speciesfilegroup/taxonworks

Workbench for biodiversity informatics.

Overview

TaxonWorks is a web-based workbench designed for taxonomists and biodiversity scientists. It provides tools to help you capture, organize, and enhance your data, collaborate with others, and prepare your work for analysis and publication. With TaxonWorks, you can easily manage your research data, share it with colleagues, and streamline the process of analyzing and publishing your findings.

To get more information on the project, its vision, and scope see taxonworks.org and docs.taxonworks.org.

License

TaxonWorks is open source and is presently available under the University of Illinois/NCSA Open Source License, read more here.

Funding

The foundation of TaxonWorks is funded by an endowment of the Species File Group. This project was funded in part by NSF-ABI-1356381. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

taxonworks's Issues

Some references to collections of objects can stall the UI

Moved from TW-376.

These cases of looking for collecting events may trigger this problem:

• /geographic_areas/show has a link to ‘Collecting event within’.
• /tasks/gis/nearby, depending on distance.
• /collecting_events/show has a link to ‘Nearby’, which uses 5000 meters for the ‘within’ distance.

It should be noted that, in the cases of

• /tasks/gis/taxon_names_distribution_data/1 and
• /tasks/gis/report
  this problem used to occur, but was circumvented by pagination of the output.

task - define an EoL submission pipeline per project

Migrated from TW-13:

Discussion with Nathan Wilson re SF2EoL.

General Issues

• Generating an DwC archive, timeouts, background generation
• IPT generation? Should we use IPT as the proxy layer to EoL?
• License issues- seem to be fairly well resolved on our side

Resolvable soft-validations

Raised by various users and developers.

   When I see a soft validation
      and it has a resolvable attribute set
        then I should see a link beside it to resolve (fix) it

Potential steps to implement:

• Straightforward, but not very integrated - add an attribute to soft_validation instances that points to route, this end point of this route is a form that lets the problem be resolved
• Integrated, but trickier - provide a mechanism to popup a form inline, the convention for defining these popups (partial name, RESTful request etc) would have to be worked out before hand. Would perhaps require multiple new attributes within the soft validation, though we should try to minimize this. Could likely take advantage of a set endpoint and a global_id.

LepIndex - references with multiple authors do not appear to have been migrated

Sources with multiple authors appear to be absent e.g. ‘Ballmer & Pratt 1992’, ‘Basquin & Cadiou, 1986’, ‘Behounek & Hacker, 1990’, ‘Beck, Kobes & Ahola, 1993’.

tag popup

The behaviour should be more or less identical to standard mobile features that allow tags to be linked to data. The main difference is that we require a definition to be provided, so the user must match a pattern (see below). Wireframes are at the end of our InVision project https://projects.invisionapp.com/share/2G5K1Z39M#/screens/209978896.

Controller code goes in the tags and keywords controller, almost certainly some tweaks to both tag and keyword model are required, and perhaps the tags concern (to handle global_id as an attribute, if it's not already there).

if I use tags_helper#new_tag_tag(object) 
   then a link/icon appears in the view
      that link is 
         wrapped in a div
           that has a unique UUID for the dive (js timestamp perhaps), *think* this is necessary for downstream interactions, might be wrong
           that references the object by object#global_id
         optionally styled with a class that indicates that this object already has tags

when I click that link
   then I get a popup text box,
     with a prompted instruction inside
     or a list of existing tags on that object
     with a undo icon to the right, that aborts *all* actions
   
when I type some letters
  then those letters get a background color bubble
  then I search through Keywords#name and #definition for a match, via a keywords_controller method
     when I find matches then I display them below the colored bubble
        when I click on a match
            then I display that match in the bubble
               with an (x) at the end of it for deletion of that keyword
            then I move the cursor to the right of the bubble
            then I inject the hidden fields into object#tags_attributes{} necessary to create a new tag
     when I can not find a match
       then I keep typing
          if I type a phrase like /(keyword\s*)+\s*:\s*some definition for foo\s*,/ (term : definition ,)
             then I create a new bubble, hiding the definition, shown only when I mouse back over (tooltip style behaviour)
             then I append hidden fields to create a new keyword AND tag (tags_attributes: [{keywords_attributes: {name: value, definition: value}}, ... ]
             and I move the cursor for the next keyword

when I find or "create" a new tag
  then I see a "create" button to the right
    when I click this button
      then an call gets made to save the data
         if it succeeds
            then a message is pulsed inside the text area
               and the popup is collapsed 
         if it fails
            then a message is pulsed inside the popup
               and the popup remains open

LepIndex - sources search only returning 5 results

The search results for sources is restricted to 5 matching sources. If there are more than 5 matching sources then the user may not have the option to select the source they are looking for.

LepIndex - Can images be ordered so the front image is the first?

Currently there seems to be a consistent ordering of the back image first. Can this be reversed so the front image appears first?

task - new georeferenced collecting event from geo-tagged image

Migrated from TW-250

This is a stub.

when I upload an image to the form
   and that image is geo-coded
     then I create the image
        and I create a new collecting event
        and I copy the geo location to the pertinent verbatim_ fields in collecting event
        and I use the option to create a verbatim georeference

Alternately, we create a new Georeference subclass. ByGeoCodedImage. This might be preferable.

unify existing #download code in module or concern

Was Jira TW-388. All the existing download code is nearly identical, make a lib/download library or model/concern and include the download code there.

allow user to customize number of items to display in list pagination

Will have to be bound to some reasonable number of records.

fix loan->show view layout

Something being duplicated? See the INHS collection project for an example.

LepIndex - duplicate Sources

For some sources there are duplicates e.g. ‘Bales 1909’, ‘Barragué, 1954’, ‘Bernardi, 1953’. The first one created on 11/09/2016 and the second record created on 22/09/2016 (the only difference is that the later version is marked up as ‘This source is used in another project.’).

CollectionProfile: new - default values, new containers, labels

As per feedback from Michelle:

When I create a new profile
   Then before I submit I can click a lock to default that value to the next record
    Then I can select the option to create a new container
    Then I can select the option to create a new Identifier for that container
    Then I can select the option to generate (que?) a Label for that container

Erroneous genera sitting under ‘Lepidoptera’ in the taxonomic hierarchy

Under ‘Lepidoptera’ there 39 genera which should not be there. Looking at the original data I can’t see any obvious data issues that explains this. If it’s just a few we can correct manually but potentially for the full dataset it could produce many more. Is it possible to track down where there were generated from?

TaxonWorks_Hierachy.xlsx

Taxon Names: Names soft failing with "Source is missing" cannot be set a source.

Steps to reproduce:

1. Create a new genus by only filling parent, name and rank Genus (ICZN).
2. Edit the name, and on source select a source and save.
3. Message indicating success will appear, but the soft validation will persist and the source will continue to be blank.

Live example: http://sandbox.speciesfile.org/taxon_names/593721 (orthoptera_species_file project). Attempt to set the source with the first result for "Cigliano".

BTW, http://sandbox.speciesfile.org/taxon_names/593720 was allowed to be created despite the genus already exists ( http://sandbox.speciesfile.org/taxon_names/369908 ).

Lepindex - citation pages have not been migrated

The citation pages (BUTMOTH_SPECIESFILE_REFS_UNIQUE.PAGE) have not been migrated to the Sources records.

LepIndex - Infrasubspecific names with wrong authorship

The valid name as shown under relationships is displaying the authorship of the name of record and not the senior name. Examples:

(1) “scheidenbaueri Gschwandner 1924” is an invalid name with the senior name as “Deilephila elpenor Linnaeus 1758”. In TW it shows the senior name as “Deilephila elpenor elpenor Gschwandner, 1924”.

(2) “costimaculata Bang-Haas 1938” is an invalid name with the senior name as “Marumba quercus [Denis & Schiffermüller] 1775”. In TW it shows the senior name as “Marumba quercus Bang-Haas, 1938”.

There are over 6,000 infrasubspecies names in the full dataset.

respond_to specs not actually testing?

Hi,

Is anyone else having this problem? If I change any of the base method specs for NomenclaturalRank with the following code:

expect(respond_to(:no_way_this_method_can_exist)).to be_true

I get:

Finished in 5.96 seconds
231 examples, 0 failures, 103 pending
[Coveralls] Outside the Travis environment, not sending data.

If you are having the same problem there then we need to rectify all specs.

NeXML?

Hi all,

the nexml.org website lists mx as one of the toolkits that is NeXML-capable. Our understanding is that mx is being deprecated in favour of TaxonWorks. The TW white paper specifies that NeXML-import is going to be enabled. Is this already the case? Does it make sense to replace the reference to mx with one to TW on our website? Is there a logo that should be used?

Thanks,

Rutger

Add confidence annotation to Annotations in show menu

Label model, Label Factories

Labels are a cache of labels waiting to be printed.

LabelFactories generate labels, they can also generate associated Identifiers.

LepIndex - Can the TaxonNo field be migrated as a data attribute field to the taxon name record

It would be a good idea to have the LepIndex TaxonNo field migrated as a data attribute field to make it easier for us to tie up legacy records once the migration is complete.

Default project-specific hub interface configuration

This was envisioned by both MM and DM.

Task and data tabs within the hub are currently defaulted to the configuration defined in https://github.com/SpeciesFileGroup/taxonworks/tree/development/config/interface/hub. Projects should be able to provide there own .yml configurations for each of data/tasks.

Rather than provide a user-interface to all the myriad options, at present the easiest way to do this is to allow a project admin to load their own .yml files, and we persist these as jsonb hashes in the @project record. Since the configuration is just a hash it will be trivial to modify the hub code to use the project has rather than the TW default. The major code needed is a validator(s) for two yml files. With them in place we just need a file-field form that has a setter that writes to the jsonb project attribute.

Steps to implement:

• Write module that validates user_tasks.yml in general
• Write module that validates data.yml in general
• Add migration to create jsonb attribute columns data_yml_settings and user_tasks_yml_settings (requires postgresql 4.?, this is OK, we need jsonb for core features)
• Create project admin form to accept user-defined data.yml and user_tasks.yml. Show user validation errors if files not valid.
• Write a application level method that returns the hub configuration, defaulting to user provided values if provided, falling back to existing methods if not

source autocomplete - improvements to behaviour and response

Todo:

• use a style to indicate whether source is in project, or not
• return a reasonable number or results
• provide example searches in data-help, and perhaps as tooltip

Task - Compare 2 records of the same type side by side

ctrl shortcuts not working

I tried all the shortcuts on this page:
....org/tasks/accessions/breakdown/sqed_depiction/978
On previous pages it would work once - the first time i tried it - but then not again
Mac OSX in Chrome

LepIndex - Can the link to the depiction launch the fuller sized image directly

On a taxon names record when selecting a depiction it navigates to an intermediate resolution of the image (depiction_1.jpg), requiring a further click to see the fuller version of the image (depiction_2,jpg). Can the first selection navigate immediately through to the fuller resolution. It's an additional unnecessary click.

Reason for not including db/schema.rb in source control?

I've noticed when I pulled from github that db/schema.rb is missing. While running "rake db:migrate" solved the issue, the comments of this file says the following:

Note that this schema.rb definition is the authoritative source for your
database schema. If you need to create the application database on another
system, you should be using db:schema:load, not running all the migrations
from scratch. The latter is a flawed and unsustainable approach (the more >migrations
you'll amass, the slower it'll run and the greater likelihood for issues).

It's strongly recommended that you check this file into your version control system.

Was this exclusion intentional? The (unspecified besides slowdowns) potential problems mentioned in the comment don't affect this project?

add Lithographic module

From TW-14. This was a reference to SF features, that got bumped, and bumped.

LepIndex - Better field titles for data attributes

For those fields that do not map directly to TaxonWorks fields they have generated 'data attribute' fields. Understandably you use the column headers we provided. However these are not very user friendly. If we provide a list of more user friendly titles can you please use these instead e.g. ‘TS_COUNTRY‘ = ‘Type Species Country’. We can send a full list through.

deactivate (prevent login and password reset request) by users with User#is_deactivated = true

This would basically be a global ban in this scenario, locking out a given account. Thoughts?

When I am a user
    and my account is flagged as is_deactivated
        then when I try to login 
             then I can not
        then when I try to request a password reset
              then I can not (fails silently)
When I am an administrator
   then I can set the is_deactivated flag to true
When I am a project administrator
   then I CAN NOT touch this flag

Provide recommendation on sharing Dockerfile or image for Taxonworks

I've added a Dockerfile for Taxonworks in my fork and blogged instructions for getting started. I suppose the next step would be to share it as an image repository, similar to https://github.com/sameersbn/docker-redmine. Any recommendations or feedback appreciated. Thanks.

style/tasks/collection_objects/filter/index (Collection objects by ...)

Will take a couple iterations to get this figured out, feel free to take artistic liberty, but to start:

• Make a nice button to toggle the area selector between map and search box.
• Make a nice toggle to toggle between slider and calendar
• Make the middle column (with only "find") have the find button float in the middle of the search filter
• Left justify headers
• There are three "facets", i.e. search buttons, make the division between each one clear
• Calendar smaller.
• Make the toggle in the results b/w map and table "nice" :)

Task - Image set comparator/light table

From JW:

Often we need to compare standard sets of images across O.T.U.’s like the montages K. has put together for some cryptic species of Aphelinus (attached).

What would be nice, is to simply drag and drop labeled images into a workspace where they could then be sorted, like slides on a light table, and printed to pdf or hard copy.

So this is a very common request. While at OSU I proposed/pitched more or less the same and Joe Cora built Specimage as a result. Heraty also had features in the proprietary package he was using. We can follow ideas therein if it seems useful- I think we might need something a little more free-form ultimately, see below.

This also bleeds into the super-deluxe matrix-evolver-note-taker thing that we are planning ot build. In that scenario you are in exploration mode, and you simply drag new images into the table as you make observations. More on that in the months to follow.

Rough proposal:

When I have pre-defined set of images (e.g. one for each of two OTUs)
   and I choose to display them on a light-table like viewer
      then I get side by side comparison of images with similar metadata
      or I can drop them all on a light-table 
          and I can drag an image in x, y
          and I can move an image forward or back in a stack
          and I can rotate an image
          and I can scale an image
     then I can draw to select a set of images
        and I can quickly annotate them into other sets
        or I can add metadata
           I can add a note
           I can add a tag
           I can change the depiction target (e.g. specimen or OTU)

Invite users by email list

We're at the point where we need to invite multiple users to the sandbox and our projects at once.
I imagine the implementation should be something like:

When I am an administrator (or a project manager)
     Then I can see 'Invite users' link in the applicable place
         When I click this link
            Then I can paste a standard email list and click to invite those people
                 When a user clicks an invite link
                     Then they spawn a new user
                          And they are asked to fill in pertinent details
                          And they are added to a project if request came from a project

Other features to consider:

• Whether to create new users on invite
• Length of validity of invite
• Length of validity of unclaimed account (if we create on send)
• How to cleanup unclaimed/validated accounts

report - collection object history

From TW-412.

Add collection_objects_controller#history.
The report should be visible through Reports.

Combine all the data below, and list them, in short form, in chronological order:

• Collecting event, date of collection
• Loan item -> loan -> date loan made
• Loan item -> loan -> date item returned
• Taxon determination -> date of determination

For any of the above with no date (in particular taxon determination), put them at the end of the list.

Refactor/renable recently visited side tab

Was TW-252. Known issues:

• Limit recent list to current project
• Restrict recent list to opt-in routes/actions per controller
• Display helper for differences b/w RESTful endpoints and task endpoints.

LepIndex - LepIndex title field is the journal/book title not the article title

The LepIndex FULLTITLE field has been migrated to the TaxonWorks article title. The field is actually the journal or book title (mostly journal title) and never the article title.

Provide link out on nomenclature browse to OTU content

Create a quick path to seeing everything that is non-nomenclature based for that name.

download basic DWCA archive (observations) for set of collection objects

Provide a scope, array of CollectionObjects, or collection_object_ids, and return DWCA.

Brief steps;

• Complete dwc-meta gem such that we can generate dwca_cache table derived from the JSON here http://gbrds.gbif.org/registry/extensions.json
• Create the DWCA index table and model
• Create DWCA meta/eml templates that will wrap descriptions
• On request of a DWCA variously generate missing (uncached) values, or pull directly from the cache

unable to destroy taxon

Was TW-397. This requires some discussion as to the behaviour of a destroy request-

Considerations:

• If there are descendants, what happens to them? Are they moved to the parent of this taxon? Also destroyed?
• What happens to the OTU that only has a link to the taxon name? Possibly- take that cached name for that taxon and put it in Otu#name.
• Do we allow for a delete when annotations and related data would destroyed as well (e.g. TaxonNameClassifications, TaxonNameRelationships, Tags, etc.), or should we require all related data to be deleted first, then allow deletion?

Allow project to default to one of ICN/ICZN

New interfaces (internal Jira 408) anticipate restrictions to one nomenclatural code or another. This could largely be done in a very smart way, wherein the parent name in the hierarchy is used as the reference nomenclature code. Simply setting the first taxon below Root to be an :icn would have the cascading effect needed. Alternately a project level setting could be added.

Steps to implement:

• Add a project attribute nomenclature_code_setting that takes one of nil, 'icn', iczn' (or other codes as they come online) OR determine code per view based on context of the "parent" taxon
• In pertinent tasks (not necessarily data views, but maybe), check for the setting and display/filter accordingly.

sort list views pagination globally, not just per page

Migrate from TW-392.

task - new taxon name

See InVision mockups for overview.

https://invis.io/2G5K1Z39M#/199963912_New_Taxon_Name_-_Task_Card_Mockup

https://invis.io/2G5K1Z39M#/212983771_1

https://invis.io/2G5K1Z39M#/212983774_All_In_One-01

add Chronographic authority

Migrated from TW-26.

Task - Summarize user activity by data type created

Using a taxon name or an OTU, show the recent work that is linked to that data.

New model Attribution (copyright, editor, maker etc. assertions)

Migrated from TW-12.

This is where we handle assertions such as CopyrightHolder, Editor, or Maker for particular data instances (and perhaps their individual attributes in some cases).

This is largely sensu this https://en.wikipedia.org/wiki/Attribution_(copyright)

Considerations:

• These should not be used to make decisions as to how or where the data are shared, made visible etc., those assertions should be handled elsewhere
• We need default project level attribution settings that should follow the model as well
• We need default inferences vs. housekeeping rules if attribution metadata are not provided

LepIndex - are the better labels for 'as subject of' and 'as object of'

I'm wondering if there are there better descriptors for 'as subject of' and 'as object of'. I think this terminology may be confusing to users of TaxonWorks. I haven't got a suggestion yet but I will have a think and get back to you.

LepIndex - Original genus not pulled through for valid names

For valid names the original genus is not being picked up i.e. where original genus != current genus e.g. for “Anthela adriana” validation states “original genus is not selected”. For the Bombycoidea it only affects 71 records but for the wider recordset it actually affects 50,000 records. For invalid names it has picked up the original genus successfully.

LepIndex - searching for a source returns an erroneous record at end of list

When searching for a source the last one on the returned list does not match the search criteria (and is always a Chalcid reference). See attached screenshot examples (sources_erroneous_search_return_1.jpg & sources_erroneous_search_return_2.jpg).