hxl-team/hxlator

PHP-based version of the HXLator

HXLator

Browser-based tool to interactively convert spreadsheets into the Humanitarian eXchange Language (HXL) (see hxl.humanitarianresponse.info). The HXL project is an initiative by the United Nations Office for the Coordination of Humanitarian Affairs.

This project picks up the idea of the original HXLator developed during the International SpaceApp Challenge 2012 in Lausanne.

Based on PHP, using Bootstrap (Apache License 2.0), EasyRdf (BSD-3-Clause license) and PHPExcel (GNU LGPL license).

Demo

A working demo of the current state of the HXLator is available at hxl.humanitarianresponse.info/hxlator.

Setup

A server running PHP 5.2.1 or higher should be sufficient to run this project. Make sure PHP can write to the uploads folder.

hxlator's Issues

Fix row headers of table (A B C …) so that the are always visible

Done? button should be available with only one row selected for hxlation

In the 'select rows' step, we may only want to hxlate a the original row that was selected during mapping. The "done" button should be available.

Reset 'mapped' flag when cancelling out of mapping.

I started to map "Government" as the source, but as it could not be looked up from the reference lists, I clicked cancel. however, the "done" flag appears on both the source property button and the cell. It is also in the Preview:

http://hxl.humanitarianresponse.info/data/displacedpopulations/1346762024042/unknown hxl:source "@lookup 1-U-5" .

This example is from the BFA 2012-05-03 dataset.

Add property definition to to the mapping popup

Currently it gives: This property should refer to a Feature from one of our reference lists. We could add a popup (can you have popups within popups?) or second line:
Feature is defined as . . .

Handling multiple sheets

Currently, a translator only works for one sheet in a workbook. If there are several workbooks, the user would have to start over again for a second sheet (at least if the structure of that sheet is different).

I'm not sure whether this is an issue for the prototype or not (I tend to think it's not).

Horizontal scroll bar issue

In Chrome and Firefox, the horizontal scrollbar for the spreadsheet viewer that appears on the page is dead (doesn't do anything). If I scroll vertically to the bottom of the datasheet, a horizontal scroll bar will appear that does work. It seems to be embedded inside the spreadsheet viewer.

The main question, "What is this data primarily about" needs to stand out more

The question text gets overwhelmed by the "Build Translator" text. The user's eye jumps directly to the top level concept boxes, but they don't see the instructions. Also, it is not clear what the numbers are on the top level concepts. Perhaps add the text "types" after the number.

Emergency / report category check

Add a function that checks whether an emergency and report category have been selected on the first page. If possible, this check should be done via JS when the user clicks the upload button, before the data is sent to hxlate.php.

The main question, "What is this data primarily about" needs to stand out more

The question text gets overwhelmed by the "Build Translator" text. The user's eye jumps directly to the top level concept boxes, but they don't see the instructions. Also, it is not clear what the numbers are on the top level concepts. Perhaps add the text "sub-categories" to this?

Scroll final RDF check window (just before submission) to top.

Current behavior was scrolled down so that the tabs couldn't be seen immediately. Once we have the table view, this should be the default.

Put the property boxes below the instructions, closer to the spreadsheet. Put a small title "Properties" above the property boxes. This also will make the collapsing of the instructions easier.

A blank value in the spreadsheet for a term that needs to be looked up results in returning all possible URIs

We should discuss what the behavior should be. Probably just to flag that this row won't be translated?

Preview table: separate multiple values per cell via " | "

IE 8 issue (if we care)

I can't remember if we decided not to support IE 8 or not. I know we had a discussion about it.

In IE8, on the first step of the mapping, the Top Level Concepts never appear.

Multiple DataContainers

Solutions:

1. Live with ValidityDates everywhere in hxlator
2. Introduce a ValidityPeriod class (hxl:TimeSensistiveData??). All classes needing a ValidityPeriod should be sub-classes of ValidityPeriod
3. Restrict certain properties from appearing in hxlator via a config file (also removes hxl:TopLevelConcept)
4. Split hxlated data into datacontainers based on unique combinations of metadata fields. Move validity date properties to apply only to data containers.
5. Dolce ontology separating temporal and spatial baseclasses.

No property allows the user to map subtypes of DisplacedPopulation (REF, OTH, IDP)

Cells are marked as mapped if the user clicks "Store Mapping" even though nothing was selected to map to.

Improve class pills loading

Minor issue: Rewrite the getClassPills function in hxlate.php so that it needs only one request to the triple store. It currently does one request for each recursion.

Stubborn user mapping data properties for what should be object properties

I'm not sure this is really an issue (at least no one that you are already aware of).

Since the APL's are not available in the lookup, I forced HXLator to map a data property for hxl:CurrentLocation with this result

. . . itarianresponse.info/data/displacedpopulations/undefined/lookup 1-B-/unknown hxl:currentLocation "@lookup 1-B-5" .

on hover over a spreadsheet cell: pop up a list of properties to which the cell as been mapped?

Not sure if this makes sense, but I'm finding that it hard to keep track of which things have been mapped to which properties.

Alternatively, we might highlight the properties that have been mapped to a cell when the cell is hovered over? That would make for an easy visual check. In fact, that is probably a better solution.

Problems in output rdf?

See the hxlate BFA 0503 First 2 rows.ttl file in the dropbox/otherstuff.

For our main example file, I hxlated just the first two rows and only f/0-5 and m/0-5.

It seems that the atLocation statements have been separated into a different population URI (lines 18-21).
The nationality statements are also in a different URI (lines 40-41).

Also, note that the APL for the second row (Dibissi) is not in the triple store (well, it's there under datatest, but that doesn't count, it's not in our real APL data). So when doing the looking for the term Dibissi, I clicked in the "start typing" box and then clicked Save mapping. The result is in line 33. The preferred behavior would be for this row to be skipped with some sort of notice to the user.

Kill the icon in the "it is a mix of these" bar.

Placeholder text in input boxes of mapping UI for lookup of reference lists

"start typing to search reference lists"

Restrict URI lookup

Restrict the URI lookup to resources that are of the correct type for the given property.

Finalize contact info

The error handler for when a submitted file is not a spreadsheet (and perhaps other places too) doesn't have any real contact info associated. We need to remember to do that later.

On upload page, remove the URI of the emergency. It just confuses people.

If we think it is useful to have it (so people can verify something about the emergency), we could have a "More information about this emergency" link.

Update URI pattern for populations

… according to the last changes in the URI patterns document. Also remove all 'unknown' slugs.

URI lookup fails if term has brackets

Make headline (build translator…) smaller

Smaller than the instructions, maybe bootstrap p lead?

Add "back" button

We'll need some functionality that allows the user to go back to the previous step in the mapping. Ideally, this should also support going back through the browser's back button, so we might have to use history.js or the like.

Break down step 3 (the actual mapping) further into numbered sub-steps

Each on its own line.

Check modal input before submission

Perform a check on whether all fields have been filled in before the modals are submitted.

Age Category Lookup issue

If we have two AgeCategorySets with the same name for one of the categories, I think it will be impossible to tell in the URI lookup window which one is from which group. How to handle that?

Show error message from incubator triple store if upload fails

There is no feedback yet if the upload does not work for some reason. The previewModal also has to be reset in that case.

After selecting a cell and property, then clicking back and re-picking the population type, the user sees the blue "Please select the cell that contains the value . . ." box even though she is on the "Select the first row of data" page.

error in generated URI for ReportCategory

Generates a URI with an unwanted space:

".../data/reportcategories/humanitarian profile"

atLocation property does not always show

e.g. when selecting the AFL class on the first page, "atLocation" does not show in the list of properties, even though it is a property of hxl:BaseClass, which is a superclass of hxl:AFL. Probably an issue with the SPARQL query in properties4class.php.

RDF Output Error

There are lots of things I don't quite understand in the output, but this one definitely looks like an error to me (POC-8 vs POC-80):

http://hxl.humanitarianresponse.info/data/displacedpopulations/bfa/UNHCR-POC-8/unknown hxl:currentLocation http://hxl.humanitarianresponse.info/data/locations/apl/bfa/UNHCR-POC-80

Popups with property definitions stopped working after the one (or possibly a few) mappings.

They came back later. Maybe something with the back button triggers this and then is reset after the next mapping is stored?

No feedback given if a user saves mapping when no URI is found during lookup

In trying to map source to "Government", this value does not exist (yet) in our reference lists. Logically I should click cancel, but if I'm stubborn and click "Save Mapping", there is no feedback. And the naked turtle shows a lookup for this property (not sure if that is a error or not).

Perhaps we need to activate and deactivate the button when it is reasonable for a user to Save or not Save the mappings?

Make lookup modal wider

Language tweaks

A few language tweaks are at: https://docs.google.com/document/d/1GK-wQgZJ_TvhjFVrCixAUDDMNDAbP4N0V824C0n4pMY/edit

Fix URI generation for AFL's

Appeared after rename from hxl:Camp.

"Done" flags on Property buttons no showing

I only had time to check this in Chrome. If I remember well, you mentioned that you had finished this, but I don't see them.

Language tweak

Change “any” to “a” in A simple online tool to convert a spreadsheet . . .

Make the various build translator instructions into numbered lists (step 1, step 2, etc.)

Hitting CTRL brings up mapping modal, even if it has not been hidden before

Make it possible to view the spreadsheet while entering data in a translator-building popup.

"No results" notice if nothing is found during lookup from reference lists

When looking up terms, some indication of which column he term came from is needed

Many times a place name may be both an admin unit and an APL. Knowing which column it came from will help the user pick the right reference.