Comments (7)
@jsimons9 let me know if you need more info
@jhammock please let us know if there's updates on the DarwinCore format that you've shared with us some time ago.
from globalbioticinteractions.
Is the DarwinCore formats you speak of at this website?? http://rs.tdwg.org/dwc/terms/index.htm
If so, I see some major difficulties. There are certain terms that we do have in common, but in some cases they are utilizing them differently. I don't know how to tackle this. I think the time and effort to try to conform completely to Darwin Core may not be possible at this time. Looking at what they have done, it appears to be a daunting task that really would require some experts and a lot time to do. The Darwin Core seems to be specifically for taxonomic collections. I will continue to study it but I would like a Darwin Core expert to look at our data categories and comment or make suggestions before we go any further. They may see it in a different light than I do.
I was once part of an effort to transform the CF database to an SQL database and that took an SQL expert and a number of CF biologists in a room together going through item by item on how to convert or group or whatever was necessary to make the conversion, and it took a number of sessions to complete.
from globalbioticinteractions.
Hi, Jim! No, the extension we are roughing up at EOL is not yet documented in the DarwinCore docs as far as I know. I think we're maybe considering publishing it eventually, but not before you guys have gotten your hands on it and had your say. We don't even really have documentation for it yet, but I do have a template file. (some new metadata fields since last you saw it, Jorrit). These are the elements we have so far:
AssociationID, Occurrence ID (required), Association Type (required), Target Occurrence ID (required), Determined Date, Determined By, Measurement Method, Remarks, Source, Citation, Contributor, ReferenceID
http://eol.org/schema/associationID, http://rs.tdwg.org/dwc/terms/occurrenceID, http://eol.org/schema/associationType, http://eol.org/schema/targetOccurrenceID, http://rs.tdwg.org/dwc/terms/measurementDeterminedDate, http://rs.tdwg.org/dwc/terms/measurementDeterminedBy, http://rs.tdwg.org/dwc/terms/measurementMethod, http://rs.tdwg.org/dwc/terms/measurementRemarks, http://purl.org/dc/terms/source, http://purl.org/dc/terms/bibliographicCitation, http://purl.org/dc/terms/contributor, http://eol.org/schema/reference/referenceID
We have draft definitions and so forth, but still- no documentation yet, and not published. (and the eol.org/schema uris don't resolve, sorry!) Still feeling our way.
from globalbioticinteractions.
Hi Jen,
Thanks for that information. Have you seen a copy of the column headers I am using for the GoMexSI data?? I would like to start serving all that data in csv outputs but not sure about waiting until we have compliance with Darwin Core or EOL. I would really be great if a group of us could get together in the same place to hash this out, but I don’t see that happening any time soon.
Are those live links in your email?? I tried the first one and got the EOL website but it said “Not Found” on the page.
Take care,
Jim
from globalbioticinteractions.
@jhammock thanks for jumping in
@jsimons9 You can find the archive using the Jen's template in the usual data access section on the GloBI wiki: https://github.com/jhpoelen/eol-globi-data/wiki#accessing-species-interaction-data . Although the data format is pretty self explanatory, I'd be happy to spend some time with you to help understand the format that the EOL team has formulated. . . actually . . . this is perfect timing to make suggestions for changes / enhancement since the format is still very much evolving. Having two separate formats is sometimes pragmatic (but somewhat expensive in the long term), but I think we should at least understand the differences between the two by answering the questions: a) how does the one format translate to the other? and b) what is lost in translation? . These two questions should be answered in the gap analysis.
Thanks for being patient!
from globalbioticinteractions.
Sorry for the delay, gents, I just took a manufactured long weekend. Jorrit, in case it matters, we just added some new optional elements to the Associations extension, mostly attribution stuff, all fields that we have for other data and had neglected to make available directly in Associations. And a few optional fields have also been added to the Occurrences sheet, to better accommodate museum collection data. The full list is now:
Occurrences:
http://rs.tdwg.org/dwc/terms/occurrenceID http://rs.tdwg.org/dwc/terms/taxonID http://rs.tdwg.org/dwc/terms/eventID http://rs.tdwg.org/dwc/terms/institutionCode http://rs.tdwg.org/dwc/terms/collectionCode http://rs.tdwg.org/dwc/terms/catalogNumber http://rs.tdwg.org/dwc/terms/sex http://rs.tdwg.org/dwc/terms/lifeStage http://rs.tdwg.org/dwc/terms/reproductiveCondition http://rs.tdwg.org/dwc/terms/behavior http://rs.tdwg.org/dwc/terms/establishmentMeans http://rs.tdwg.org/dwc/terms/occurrenceRemarks http://rs.tdwg.org/dwc/terms/individualCount http://rs.tdwg.org/dwc/terms/preparations http://rs.tdwg.org/dwc/terms/fieldNotes http://rs.tdwg.org/dwc/terms/samplingProtocol http://rs.tdwg.org/dwc/terms/samplingEffort http://rs.tdwg.org/dwc/terms/recordedBy http://rs.tdwg.org/dwc/terms/identifiedBy http://rs.tdwg.org/dwc/terms/dateIdentified http://rs.tdwg.org/dwc/terms/eventDate http://purl.org/dc/terms/modified http://rs.tdwg.org/dwc/terms/locality http://rs.tdwg.org/dwc/terms/decimalLatitude http://rs.tdwg.org/dwc/terms/decimalLongitude http://rs.tdwg.org/dwc/terms/verbatimLatitude http://rs.tdwg.org/dwc/terms/verbatimLongitude http://rs.tdwg.org/dwc/terms/verbatimElevation
Associations:
http://eol.org/schema/associationID http://rs.tdwg.org/dwc/terms/occurrenceID http://eol.org/schema/associationType http://eol.org/schema/targetOccurrenceID http://rs.tdwg.org/dwc/terms/measurementDeterminedDate http://rs.tdwg.org/dwc/terms/measurementDeterminedBy http://rs.tdwg.org/dwc/terms/measurementMethod http://rs.tdwg.org/dwc/terms/measurementRemarks http://purl.org/dc/terms/source http://purl.org/dc/terms/bibliographicCitation http://purl.org/dc/terms/contributor http://eol.org/schema/reference/referenceID
This need not have any effect on your data, unless there are cases where you felt detail was missing previously. Let me know if you're curious about any of the new fields.
Jen
from globalbioticinteractions.
After some in-person discussion with Jim, I figured that the GoMexSI format is optimized for data entry, while the darwin core archive is optimized for exchange of large datasets between bioinformatics systems. This is why I believe that a gap analysis between the two formats is no necessary. Instead, we'll continue to effort to ensure the quality of the data mapping from GoMexSI format to normalized data elements that GloBI produces. Closing issue.
from globalbioticinteractions.
Related Issues (20)
- index WorldFAIR catalog available through https://github.com/mtrekels/WorldFAIR HOT 2
- link names to their related MCZ types specimen HOT 3
- Occurrence associations from NEON Biorepository HOT 3
- review Jen Yost's OBI Herbarium collection for GenBank accession links HOT 20
- suggest to index Gerstner, B.E., P.L. Zarnetske, and P. Bills. 2023. Frugivoria: A trait database for birds and mammals exhibiting frugivory across contiguous Neotropical moist forests ver 5. Environmental Data Initiative. https://doi.org/10.6073/pasta/168e95f04d4726d31d868bfe22d749a5 (Accessed 2023-06-13). HOT 1
- suggest to index https://darkcides.org/database/ a bat roosting dataset HOT 4
- Provide labels for Wikidata identifiers
- suggest to index https://darkcides.org/database/ HOT 1
- add button to locate "applicable policy" for specific species interaction
- HOSTS - a Database of the World's Lepidopteran Hostplants HOT 10
- suggest to index IPBES documents for species interactions as well as taxonomic names HOT 15
- introduce html review pages HOT 6
- "no checkpoint found" error in neo4j causes intermittent search index issues HOT 1
- search by occurrence id even if another external id is present
- add support for TSN: taxonomic identifier prefixes HOT 1
- publish GloBI review to RIO journal via recently created jats review report HOT 2
- add summary statistics to review reports HOT 3
- suggested to index Schwarz, Benjamin et al. (2021). Data from: Temporal scale-dependence of plant-pollinator networks [Dataset]. Dryad. https://doi.org/10.5061/dryad.qz612jmbp HOT 6
- error when streaming interactions.tsv.gz into mlr : mlr: CSV header/data length mismatch 92 != 42 at filename (stdin) line 294204. ] HOT 6
- Making a dataset for review based on a query
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from globalbioticinteractions.