databrary/databraryr

databraryr

Databrary

Want to help out? Please reach out via email, commenting on an issue, or whatever works for you. We have significant experience mentoring new developers and Haskell developers.

We have preliminary library documentation on our GitHub Pages site. Haddocks coverage would be another excellent place to start contributing...

A simple code coverage report is also available on our GitHub pages site.

Why?

Accelerate the pace of sharing and reusing recorded, annotated video, to increase the pace of analysis and discovery in behaviorial science. Sharing restricted videos without burdening researchers heavily requires a sophisticated, yet simple, approach to entering release level information and authorizing networks of users to datasets. It also requires convenient mechanisms for uploading large collections of video files. It requires allowing for a very flexible model of meta-data to allow a large variety of attributes to be attached to a given dataset.

Demo

Go for a test drive on http://dev1.databrary.org:8000/. We are excited to have your attention and interest.

1. Login as [email protected] / unastan54
2. Click on "Testarosa Tesla", in the upper right corner, and select "Create Volume"
3. Select "No" at the bottom of the page and enter some text in boxes provided
4. Click on the "Enter Data" tab
5. Click on "add folder", enter a test date, e.g. "01/02/2017"
6. Scroll to the top of the page and click the "eye" icon, to switch to view mode

Try browsing around a site with example data here: http://sandbox1.databrary.org:8000/

You can get an understanding of the full scope of features offered in our feature catalog (pdf). Some pages have changed since this catalog was produced.

Related Work (Open Source)

• https://github.com/IQSS/dataverse
• https://github.com/DSpace/DSpace
• https://github.com/CenterForOpenScience/osf.io

Testing, Requirements, Implementation Documentation

	docs
functional	docs
internal routes
front end
back end

	results	coverage
functional
front end integration
front end unit
back end integration
back end unit

Upstream - Anticipating and Contributing

Anticipating features and libraries:

• aeson JSON parsing with accumulative errors
• smtp-mail issue
• zip Blind Archive Support
• beam
• servant-swagger
• hedgehog State machine testing multiple results
• ldap-haskell

Recent contributions:

• https://git.coop/lwm/tasty-discover/issues/145
• https://github.com/lspitzner/brittany/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+chreekat

Service and Operations Status

Nix Cache -

Open Source Report Card

We try our best to use open source throughout our source code, our development and our operations. Here is where we stand currently.

Open source tools: ghc, all haskell libraries, nixpkgs, centos, ubuntu, nixos, redoc/swagger, solr/openjdk, postgresql, nginx, vim, emacs, slurm.

Non-open source tools: runscope, katalon, github, jira, confluence, aws, macos, mailtrap, gmail, slack, google drive, skype, sublime.

License

Copyright (C) 2013-2023 New York University

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

databraryr's Issues

Address to-dos from CRAN review

Thanks,

Please always write package names, software names and API (application
programming interface) names in single quotes in title and description.
e.g: --> 'Databrary.org'
Please note that package names are case sensitive.

It seems like you have too many spaces in your description field.
Probably because linebreaks count as spaces too.
Please remove unecassary ones.

Please add \value to .Rd files regarding exported methods and explain
the functions results in the documentation. Please write about the
structure of the output (class) and also what the output means. (If a
function does not return a value, please document that too, e.g.
\value{No return value, called for side effects} or similar)
Missing Rd-tags:
check_ssl_certs.Rd: \value

Please put functions which download data in \donttest{}.

You write information messages to the console that cannot be easily
suppressed.
It is more R like to generate objects that can be used to extract the
information a user is interested in, and then print() that object.
Instead of print()/cat() rather use message()/warning() or
if(verbose)cat(..) (or maybe stop()) if you really have to write text to
the console. (except for print, summary, interactive functions) ->
R/list_volume_metadata.R

Please fix and resubmit.

Best,
Benjamin Altmann

Create documentation using pkgdown

Review existing codebase

Add download_*.zip functions

Enhance search by keyword, search by tag to return matching volumes

Currently, search_for_keywords() and search_for_tags() return the keywords and tags.

Fix GitHub actions rendering of pkgdown documentation

Error looks like there is a missing dependency on rmarkdown.

Plan roadmap

Fix logout_db() bug

Session token remains when logging out.

Reconcile list_*, get_*, and download_* functions

The terminology is inconsistent.

Add NEWS.md

Change "master" branch to "main"

When we're at a good place to change our repos, we should rename the "master" branch to "main".

See: https://github.com/github/renaming

Remove redundant functions

• download_containers_records(); duplicates list_containers_records()
• download_party_json(); duplicates functionality in download_party()
• download_video() may duplicate download_asset()

Fix authentication in download_session_csv for non-public files

Evaluate feasibility of new upload_ or edit_ functions

v0.5.1

Extract function-specific tests into independent R files

This is recommended practice in https://r-pkgs.org/testing-basics.html#create-a-test

Make functions accept lists

Release databraryr 0.5.0

First release:

• usethis::use_news_md()
• usethis::use_cran_comments()
• Update (aspirational) install instructions in README
• Proofread Title: and Description:
• Check that all exported functions have @return and @examples
• Check that Authors@R: includes a copyright holder (role 'cph')
• Check licensing of included files
• Review https://github.com/DavisVaughan/extrachecks

Prepare for release:

• git pull
• Check if any deprecation processes should be advanced, as described in Gradual deprecation
• usethis::use_github_links()
• urlchecker::url_check()
• devtools::check(remote = TRUE, manual = TRUE)
• devtools::check_win_devel()
• git push
• Draft blog post

Submit to CRAN:

• usethis::use_version('minor')
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• Add preemptive link to blog post in pkgdown news menu
• usethis::use_github_release()
• usethis::use_dev_version(push = TRUE)
• usethis::use_news_md()
• Finish blog post
• Tweet

Build issues: "no visible binding for global variable"

Migrate analytics to databrary GitHub organization

Review authentication functions and clean-up, as needed.

R/config_passwd.R
R/db_login.R
R/db_logout.R

How to determine volume from session/slot ID

The API returns assets from session/slots without requiring the volume, but it would be useful to have the volume ID available as metadata or for the file name.

Generate critical feature list

Complete CRAN pre-submission checklist

Prepare for release:

[x] git pull
[x] Check if any deprecation processes should be advanced, as described in Gradual deprecation
[x] usethis::use_github_links()
[x] urlchecker::url_check()
[x] devtools::check(remote = TRUE, manual = TRUE)
[x] devtools::check_win_devel()
[x] git push
[x] Merge branch with main
[ ] Draft blog post
[ ] Submit to CRAN:

[ ] usethis::use_version('minor')
[ ] devtools::submit_cran()

Approve email

Advise on how to handle works-in-progress (WIP)

Some functions are still in development and not ready for primetime. How to handle this?

I've dumped them in R/_wip.R for now, but this doesn't seem like the best approach.

Complete CRAN checklist

Finish migration from old repo, name change

Finish debugging & testing login_db()

I discovered some anomalies in testing the function. I added some debugging messages, made some flag calls more explicit, and made SERVICE an input parameter. But in testing I got 400 errors, so I have to wait to finish testing.

Remove Databrary cookie at end of test()

How do we remove the Databrary cookie generated by running tests on the login_db() function?

Expand test coverage for download_* functions

Expand functionality to more closely mirror emerging schema

Move Datavyu functions to new, separate repo

dv_to_csv(), extract_dv_code_defs(), extract_dv()

When Databrary storage issue has resolved, re-run `urlchecker::url_check()`.

I suspect the calls are timing out.

Expand vignette coverage

Explore uses of av package or similar ones within databraryR

https://docs.ropensci.org/av/

Modify download_party_avatar() to download file.

Currently, download_party_avatar() displays the avatar (as a tibble!), but does not download it.

Change parameter checks to use assertthat

Upgrade README.md

Add NSF grant award functionality

Fix GitHub actions rendering of pkg_down documentation

Evaluate feasibility of replacing list_session_in_volume() calls with list_sessions()

`devtools::check(remote = TRUE, manual = TRUE)` reveals a set of HTML issues with the documentation files.