Giter Site home page Giter Site logo

perseids-project / digital_milliet Goto Github PK

View Code? Open in Web Editor NEW
0.0 7.0 4.0 22.62 MB

The Digital Milliet

Home Page: https://digmill.perseids.org

Python 8.49% CSS 26.31% HTML 4.59% JavaScript 53.04% Makefile 0.06% Puppet 0.30% Shell 0.04% Dockerfile 0.04% SCSS 7.14%

digital_milliet's Introduction

https://travis-ci.org/perseids-project/digital_milliet.svg?branch=master https://coveralls.io/repos/perseids-project/digital_milliet/badge.svg?branch=master https://readthedocs.org/projects/pip/badge/?version=latest

Full Documentation at http://digital-milliet.readthedocs.io/en/latest/

Overview

The Digitat Milliet supports the creation and display of an interactive collection of ancient Greek and Latin texts about painting. It is a digital interpretation of "The Recueil des textes grecs et latins relatifs à la peinture ancienne" (“Collection of Greek and Latin Texts Concerning Ancient Painting”), the initiative of a French academic painter, Paul Milliet, who had a passion for ancient Greek culture.

https://github.com/perseids-project/digital_milliet/blob/master/doc/dmhome.png?raw=true

https://github.com/perseids-project/digital_milliet/blob/master/doc/dmbrowse.png?raw=true

https://github.com/perseids-project/digital_milliet/blob/master/doc/dmread.png?raw=true

The Digital Milliet is implmented as a Flask Application, backed by a MongoDB database, supported by external web services.

Installation Instructions

The following instructions are for setting up a Development environment for Digital Milliet.

Install Prerequisites:

  • mongodb
  • python 3.5, pip and virtualenv
sudo apt-get install -y python3-pip python3-dev build-essential mongo

Clone the repository

git clone https://github.com/perseids-project/digital_milliet

Setup the sample data

mongorestore digital_milliet/db/sample

Create a virtual environment

cd digital_milliet
virtualenv -p /path/to/python3 venv
source venv/bin/activate
python setup.py install

Run the code, installing test fixtures and with a fixed user:

python runtest.py --install --loggedin

Or with Docker and Docker Compose

git clone https://github.com/perseids-project/digital_milliet
cd digital_milliet
docker-compose build
docker-compose up

For production deployment, see Puppet manifests in the puppet subdirectory of this repository.

Configuration

All deployment specific variables and dependencies are specified in an external configuration file. By default the application looks for a configuration file named config.cfg in the digital_milliet base directory. An alternate path can be supplied in an argument to the DigitalMilliet Flask Application:

DigitalMilliet(app, config_files=["path/to/your/config.cfg"])

The default contents of this configuration file, with explanation of each setting, is provided below:

# Name of the Mongo database
MONGO_DBNAME = 'app'

# Secret key for Flask session
SECRET_KEY = 'development is fun'

# Perseids OAUTH Setup
# OAUTH_CONSUMER_KEY and OAUTH_CONSUMER_SECRET must be supplied by Perseids Administrator for Production use
OAUTH_NAME = "digitalmilliet"
OAUTH_CONSUMER_KEY = 'dummy'
OAUTH_CONSUMER_SECRET = 'dummy'
OAUTH_REQUEST_TOKEN_PARAMS = {'scope': 'read'}
OAUTH_BASE_URL = 'https://sosol.perseids.org/sosol/api/v1/'
OAUTH_ACCESS_TOKEN_URL = 'https://sosol.perseids.org/sosol/oauth/token'
OAUTH_ACCESS_TOKEN_METHOD = "POST"
OAUTH_REQUEST_TOKEN_URL = None
OAUTH_AUTHORIZE_URL = 'https://sosol.perseids.org/sosol/oauth/authorize'
OAUTH_CALLBACK_URL = 'https://digmill.perseids.org/digmil/oauth/authorized'

# Name of the collection for author records (future proofing to enable move to a separate collection)
AUTHORS_COLLECTION = "annotation"

# Set this to the ID for the Perseids community id in which membership enables Digital Milliet editorial permissions
ENFORCE_COMMUNITY_ID = None

# Not to be used in Production: eases development without OAuth Setup
OAUTH_USER_OVERRIDE = { 'oauth_user_uri' : 'http://sampleuseruri', 'oauth_user_name': 'Sample User' }

# Perseus Catalog API - Used for Lookup of Author and Work Metadata
CATALOG_API_URL = 'http://catalog.perseus.org/cite-collections/api'
CITE_URI_PREFIX = 'http://perseids.org/collections/'
CITE_COLLECTION = 'urn:cite:perseus:digmil'

# CTS API Endpoint for Retrieval of Primary Source Texts and Translations
CTS_BROWSE_URL = 'https://cts.perseids.org'
CTS_API_URL = 'https://cts.perseids.org/api/cts/'
CTS_API_VERSION = 5

Authentication and Authorization

The Digital Milliet application itself does not provide a user model or any AAI functionality.

The Create, Update and Delete functionality of the Digital Milliet application can be protected by the OAuth2 protocol. The location of the OAuth2 endpoint and other details must be supplied in these configuration settings:

OAUTH_NAME = "digitalmilliet"
OAUTH_CONSUMER_KEY = ''
OAUTH_CONSUMER_SECRET =''
OAUTH_REQUEST_TOKEN_PARAMS = {'scope': 'read'}
OAUTH_BASE_URL = ''
OAUTH_ACCESS_TOKEN_URL = ''
OAUTH_ACCESS_TOKEN_METHOD = "POST"
OAUTH_REQUEST_TOKEN_URL = None
OAUTH_AUTHORIZE_URL = ''
OAUTH_CALLBACK_URL = '<digmill_application_host>/oauth/authorized'

The deployment at https://digmill.perseids.org uses Perseids (https://sosol.perseids.org/sosol) as its OAuth2 provider. Perseids in turn delegates to Social Identity providers for user authentication. Perseids assigns a URI identifier to authenticated users and users supply a public-facing full name that they wish to be affiliated with their Perseids account. This information (the Perseids User URI and Full Name) are added as the creator associated with annotations created in the Digital Milliet application. Once a record is created, if it's edited by a user other than the creator, that user is added as an additional editor in the updated annotations.

Although not recommended for production use, it is possible to disable the OAuth2 protection by setting the name and URI to associate with all records via the OAUTH_USER_OVERRIDE configuration setting. This could be used in combination with a simpler authentication method such as HTTP Basic Authorization.

OAuth2 provides Authentication but not Authorization support. (By Authorization we mean restricting create/update/delete access of Digital Milliet entries to only specific authenticated users.) Implementing a full user model and role-based authorization was out of scope for development of the Digital Milliet application. A potential future goal is to use the Perseids platform to provide editorial review board functionality, removing the ability to edit annotations directly in the Digital Milliet application.

With this goal in mind, we implemented a Perseids-specific stop-gap solution to provide Authorization functionality to the Digital Milliet application. The application configuration allows for the specification of the identifier of a Perseids review community (via the ENFORCE_COMMUNITY_ID setting). If this is specified, then authenticated users must be a member of the Perseids Community with that id in order to be able to create, edit or delete entries in the Digital Milliet. If the ENFORCE_COMMUNITY_ID setting is left empty, this functionality is disabled and all authenticated users can create, edit or delete entries.

Design: Motivation, Standards, Dependencies

The aim behind the design of the application was to support the representation of each entry in the original "Recueil" as a graph of annotations.

The primary annotation of a Digital Milliet graph/record set is a Commentary targeting a stable CTS URN identifier of the primary source Greek or Latin text which was the subject of the entry in the "Receuil". This commentary annotation gets assigned an identifier which includes the original number of the entry in the "Receui". Throughout the code and interface, this is referred to as the "Milliet Number".

Additional annotations in each graph include a Bibliography, French and English translations of the primary source text, tags (freeform and semantic) as well as images representing the described artwork or related material. The images can also be annotated.

Entries are indexed for browsing both by Milliet Number and Author/Work/Passage of the target primary source text passage.

The Digital Milliet application retrieves Author and Work metadata for each primary source text is from the Perseus Catalog (http://catalog.perseus.org/).

We have used a non-standard form of a CITE URN to assign identifiers to each individual annotation in the graph. This may eventually be replaced by UUIDs or other identifier system.

In order to facilitate data reuse and interoperability we represent these annotations according to the Open Annotation data model (http://www.openannotation.org/), a standard data model for serializing annotations on resources in the world wide web. (This model has now evolved into the W3C Web Annotation Model). Image annotations adhere to the IIIF standard (http://iiif.io).

The original design called for primary source texts and translations to be identified only by their CTS URN identifiers and all textual passages retrieved at runtime from CTS Repositories.

However, as many of the texts and/or translations we need to refer to are not yet available online at a published CTS API endpoint, and the stability and long term sustainability of such end points are not clear, the application design was changed to enabled textual content to be included in addition to or instead of the CTS URN identifier of a text or translation.

The Digital Milliet application depends upon components of the CapiTainS suite (https://github.com/capitains) for its interaction with CTS endpoints and validation of CTS URN syntax.

The application uses the IIIF standard for image referencing and annotations and reuses the open source Mirador Viewer (http://projectmirador.org/) to provide image display and annotation functionality.

Workflow

The primary workflow for creating a new entry in the Digital Milliet is described in the diagram below.

https://github.com/perseids-project/digital_milliet/blob/master/doc/digitalmillietnewcommentaryworkflow.png?raw=true

Individual components of an entry can also be edited or added separately after the initial data entry, via the Edit interface.

To create a new entry, you click the Add Record button to bring up the Create form:

https://github.com/perseids-project/digital_milliet/blob/master/doc/dmnew.png?raw=true

Use the typeahead features in the 'Search for a Primary Source Passage' to search for an existing text in the CTS Repository

https://github.com/perseids-project/digital_milliet/blob/master/doc/dmnew2.png?raw=true

If found, you can enter the passage range you are interested in and then click 'Retrieve' to to retrieve the text.

If text you need is not found you can supply the text yourself in the input box.

Proceed to enter commentary text, tags and bibliography. Follow the same procedure for translations as you did for the primary source text.

https://github.com/perseids-project/digital_milliet/blob/master/doc/dmnew3.png?raw=true

If an image you want to associate with the entry is available in from an IIIF-compliant image server you can enter the publisher and URL of the IIIF manifest. This can be an image manifest, or a canvas manifest.

https://github.com/perseids-project/digital_milliet/blob/master/doc/dmnew4.png?raw=true

To edit an existing entry, you click the Edit button next to the Digital Milliet number on the Browse display. You must be logged into see this option.

Editing proceeds similarly to the process for creating a new entry.

https://github.com/perseids-project/digital_milliet/blob/master/doc/dmedit.png?raw=true

Image annotations can be viewed, added, edited and deleted directly using the Mirador viewer.

https://github.com/perseids-project/digital_milliet/blob/master/doc/dmimage.png?raw=true

Click on the bubble icon to view annotations on the image. Hover your mouse over the marked up areas on the image to see the annotation text.

If you are logged in you can click Edit or Delete to edit or delete the image annotation.

You use the drawing tools in the Mirador viewer to create new annotations. Select a tool and drag the mouse to highlight the region of interest on the image. When you release the mouse the annotation dialog will popup and you can enter and save your annotation text.

digital_milliet's People

Contributors

annakrohn avatar balmas avatar ponteineptique avatar zfletch avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

digital_milliet's Issues

active language tab should match language displayed

commit 22d7a4b improved language handling but we need to make a tweak to make sure the active tab gets set to the right language

Eg. if an entry has only a french translation and not an english, the english tab is active even though it's the french that is displayed.

probably a minor javascript fix is all that's required.

author search

perhaps save additional mongo docs that are just author name, works, and ids? or do I add to the current mongo docs?

Refinement of handling of multiple images

  1. it would be nice if the image viewer gave some indication of whether multiple manifests are available. I can't see anyway to determine this other than by clicking on add slot

  2. if possible it would be good to hide or remove the "Add new object from URL" option from the Mirador viewer, since we don't allow editing of the manifests in this view.

If these require changes in Mirador code, we they can be postponed to a later phase, just noting them for now.

Editing page

Create a quick means of editing submitted entries.

Don't save an empty image annotation

if you edit an entry which doesn't have images, you get a nice ui to add one, but this adds an annotation to the images [] set for the record which has an empty body (it has a dc:publisher with an empty string)

Should validate the image annotation on save and only add it if it's populated.

add a delete option

it would be good to be able to delete annotation documents from the edit ui

cleanup the commentary targets

due to a change mid development, commentary targets in the data take any of the following forms

  • a single URN string
  • an Object containing Markdown text
  • an Array of a URN string (possibly empty) and an Object containing Markdown text

The code now always creates the last of these whenever a new commentary is added, or an existing commentary is edited.

We should clean up the data so that it's consistent, and insert the URNs when they are known.

accomodate alphanumerics in cite urns for dig mill commentary

it seems its possible for invalid cite urns to be assigned to commentary items. For example, we have this:

urn:cite:perseus:digmil.76a.c1

which was causing the app to fail in process_comm. I've caught the error now (see 78cbe7f) but we need to investigate how this occurs and prevent it.

deploy at root path

now that it's a single service, we should deploy the app at the root path of the server.

document the application

Items to be documented:

  • installation
  • use cases for the application
  • CTS based data entry/edit workflow
  • mongodb schema and annotation model
  • application/deployment dependencies
  • step-by-step guide to using the digital milliet + perseids client apps to create a cts+oa+iiif publication

externalize the collection names

Author records are now stored in the db collected named according to the AUTHORS_COLLECTION config file setting. It would be nice to externalize the names of the commentaries and mirador collections as well.

Glossary for terms

Different from tags, just general terminology that some people might not know. Link to from words

Implement Tagging

Perhaps put together quick system for this as stop-gap until we can get it working in Perseids.

Linking commentary and Mirador Annotations

Where in the model would you like me to add the link between an annotation and the model ?

[{
	"@context": "http://iiif.io/api/presentation/2/context.json",
	"@type": "oa:Annotation",
	"motivation": [
		"oa:commenting"
	],
	"_id": {
		"$oid": "591ad46c87407e232f03fb97"
	},
	"on": [
		{
			"@type": "oa:SpecificResource",
			"full": "http://gallica.bnf.fr/iiif/ark:/12148/btv1b8304485j/canvas/f1",
			"selector": {
				"@type": "oa:Choice",
				"default": {
					"@type": "oa:FragmentSelector",
					"value": "xywh=5355,1538,1607,1329"
				},
				"item": {
					"@type": "oa:SvgSelector",
					"value": "<svg xmlns='http://www.w3.org/2000/svg'><path xmlns=\"http://www.w3.org/2000/svg\" d=\"M5355.44812,1625.14423l803.31731,1241.49038l803.31731,-861.74038l-686.47115,-467.38462l-920.16346,43.81731l0,87.63462l861.74038,1197.67308\" data-paper-data=\"{&quot;strokeWidth&quot;:1,&quot;editable&quot;:true,&quot;deleteIcon&quot;:null,&quot;annotation&quot;:null}\" id=\"rough_path_04b19925-49d8-4d3f-b4bf-050bd885017d\" fill=\"none\" fill-rule=\"nonzero\" stroke=\"#00bfff\" stroke-width=\"1\" stroke-linecap=\"butt\" stroke-linejoin=\"miter\" stroke-miterlimit=\"10\" stroke-dasharray=\"\" stroke-dashoffset=\"0\" font-family=\"none\" font-weight=\"none\" font-size=\"none\" text-anchor=\"none\" style=\"mix-blend-mode: normal\"/></svg>"
				}
			},
			"within": {
				"@type": "sc:Manifest",
				"@id": "http://iiif.biblissima.fr/manifests/ark:/12148/btv1b8304485j/manifest.json"
			}
		}
	],
	"resource": [
		{
			"@type": "dctypes:Text",
			"format": "text/html",
			"chars": "<p>Test 2</p>"
		}
	]
}]

Phase 2 Workplan

The Phase 2 work has the following main goals:

  1. Add unit tests and documentation to the code so that it can be packaged and reused by other projects and developers
  2. Work towards further integration into the Perseids infrastructure to take advantage of upcoming semantic annotation functionality
  3. Add support for displaying and annotating images via IIIF

Assuming we continue with current plans for the CapiTaiNs Nemo application in Perseids, the immediate term vision for the digital milliet application would be for it (and perseids-client-apps), to be refactored as a Nemo plugins. However, given that most of the work has to take place in July and that the Nemo plugin docs need further work before they can efficiently guide developers unfamiliar with the code, we need to do this refactoring in stages.

I've setup 3 milestones in the issues list which cover goals 1 and 2, and suggest we proceed in this order:

Milestone 1: Fix outstanding bugs, add unit tests of all existing functionality and framework for documentation

Milestone 2: Refactor as Flask Extension, add missing routes, and move CTS passage retrieval from client to server.

Milestone 3: Document, package, deploy

The work in Milestones 1 and 2 will lay the groundwork to ease the eventual refactoring as a Nemo plugin without tying us to that approach in the short term.

Image support will come after Milestone 3, as will continued integration with the Perseids and CapiTaiNs infrastructure. The latter will bring along user authentication and semantic annotation, both of which are actively being worked on through Perseids in parallel with Milestones 1-3 here.

Rrevent creating new annotations if user isn't logged in

it would be nice if we could hide or remove the ability for users to create new annotations on images in the mirador viewer if they aren't logged in. It seems the ability edit or delete existing annotations is alreayd triggered by that, but the new annotation features are still there. saving doesn't do anything but it also doesn't warn the user that it won't.

possible to corrupt a translation entry

Reported by Spiros:

So I logged in to review entry 289 last night around 2 o'clock in the morning, and I did it successfully in terms of correcting and amending the commentary, the bibliography and the French translation. But when I tried to correct the English translation, which, by the way, was only an e-mail address to Perseus I think and not the text of the translation, the system blogged and I received the following message:

“The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator at [email protected] to inform them of the time this error occurred, and the actions you performed just before this error.”

Looking at the database, the entry for the english translation did not conform to the correct structure. The code assumes that if the hasBody of a translation is text then it will be a CTS URN and if not, then it will be an object with chars and lang attributes. This was neither -- it was the text of the translation Spiros entered. It should not be possible for the edit form to save an invalid data structure, so this needs to be investigated.

Also noted that some of the annotation ids seem a little weird - i.e. invalid english translation had an id of digmilann.5404602 rather than a full CITE URN.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.