getodk/docs

The documentation for all the ODK tools. This is one of the most popular artifacts our community produces. It's built in Sphinx. ✨📚✨

opendatakit

The developer wiki (including release notes) and issues tracker are located here.

This site is primarily for developers. If you are not a developer, please visit https://getodk.org/

ODK is a free and open-source set tools which help organizations author, field, and manage mobile data collection solutions. ODK provides an out-of-the-box solution for users to:

• Build a data collection form or survey;
• Collect the data on a mobile device and send it to a server; and
• Aggregate the collected data on a server and extract it in useful formats.

In addition to socio-economic and health surveys with GPS locations and images, ODK is being used to create decision support for clinicians and for building multimedia-rich nature mapping tools. See featured deployments and list of tools for more examples of what the ODK community is doing. We welcome and encourage participation from the user community.

Downloads of the tools are available at Downloads.

docs's Issues

Document how to use Briefcase to pull forms directly from device

How to pull forms directly from the device and its limitations not documented:
https://forum.opendatakit.org/t/export-data-from-mobile/9126

Describe how to get bug reports off of Android

https://developer.android.com/studio/debug/bug-report.html is the ideal for Android 6+

There are also some notes at https://github.com/opendatakit/opendatakit/wiki/Collect-Troubleshooting

Getting Started Guide

Collect development : images in QuestionWidgets

If we want to use imageView, imageButton in Widgets then we should avoid adding the image to the xml layout. The image should be added dynamically using setBackgroundResource() so that the garbage collector can recycle the bitmaps and prevent the activity from crashing (caused while trying to use a recycled bitmap).

Form Design Guide

https://opendatakit.org/help/form-design/
and especially this:
https://opendatakit.org/help/form-design/examples/

But with Excel-based on examples as well as XML,
and links to the spec., etc.

Document advanced use of external apps

Custom error messages and field-lists -- getodk/collect#129 and https://opendatakit.org/help/form-design/external-apps/ (done)

Document navigation setting

We've been hearing about groups that use ODK to survey the general population. They'll do things like hand tablets around rather than using trained enumerators. One challenge they face is that people don't know to swipe between questions. It seems they should probably be using the "use swipes and buttons" navigation setting.

It would of course be helpful to have documentation on all the settings. But for this one specifically, it would be helpful if it were included in "advice for surveys without enumerators" or something like that.

Area function

It's not well-known and I'm not finding any documentation.

https://forum.opendatakit.org/t/measureheigt-area/9128/5

@adammichaelwood I'm not really sure how to tag this one. Perhaps there should be a form design label?

Video sphinx directive

Currently, images get stored at img, but videos are stored at _static/vid and it's not clear why. Unless there is a good reason, we should standardize on the location.

Automate the form widget guide from a structured XLSForm

It would be great if someone with Android/Java skills could work with me to create a script (using monkeyrunner, or similar) that would parse an XLSForm and run through the form on a device, taking screenshots and outputting the images and other info into a doc -- (replacing form-widgets.rst)

Reduce the size of JPEGs before publishing

On publication, we should run lossy compression with sane defaults on all jpgs.

I believe the best way to do this is likely to use mozjpeg. We'll likely have to install from source and https://nystudio107.com/blog/installing-mozjpeg-on-ubuntu-16-04-forge is a decent guide.

New Template, design fixes

apply RTD template, and do some other design work

Use lossless compression on all source images

Per this discussion at #64 (comment), I propose we use run lossless compression on all source images because there's no real reason to have massive source images.

Add footer with log issue

I was just perusing Kubernetes docs (as one does) and noticed that they have a handy "Create an issue" button at the bottom of the docs page (example here). I think that might be a nice way to get more people involved. Maybe since the ODK audience is less technical the link can say "there's something missing or wrong on this page" or something like that.

The cool thing is that the issue link can pass on the page that the problem is on so the user doesn't risk forgetting it.

Training videos

curate a list of ODK training videos as per @yanokwa suggestion here: https://forum.opendatakit.org/t/odk-learning-videos/9155/3

How to push forms to a phone from a Windows PC

Very useful forum post: https://forum.opendatakit.org/t/pushing-pulling-forms-to-from-android-devices-directly-from-a-pc/7016

Build some kind of mental model for external data

• "External itemsets": https://opendatakit.org/help/form-design/external-itemsets/ (http://xlsform.org/#external-selects)

  • Very narrow. Allows for building a CSV in the same structure as the choices sheet that gets imported to a database to speed up large lists.

• "Pulldata": https://opendatakit.org/help/form-design/data-preloading/ (http://xlsform.org/#pre-loading-csv-data)

  • Function that can be used in calculations to pull a cell from a CSV.

• "Search appearance-function": http://xlsform.org/#dynamic-selects-from-pre-loaded-data

  • Function used in appearance that pulls a column from a CSV to use as a choice list.

These have overlap in functionality.

The XForms spec-compliant way to do all this and be able to use XPath for querying is https://opendatakit.github.io/xforms-spec/#secondary-instances---external.

Let's punt on this until we have a rough plan of how to resolve some of these historical challenges.

Document Aggregate installation

Document aggregate installation:
https://forum.opendatakit.org/t/odk-aggregate-minimum-requirements-for-supporting-packages-java-tomcat-mysql/9345

Current documentation needs updating:
https://opendatakit.org/use/aggregate/

https://github.com/opendatakit/opendatakit/wiki/Aggregate-AWS-Install

Document range widget

What it looks like in Collect, the different appearances, how to generate one in Build or XLSForm.

getodk/collect#105

Describe admin settings menu and how it's typically used

getodk/collect#1306 - it's not all that clear

This may exist somewhere but I'm not immediately finding it.

pyxform: describe how to get the command-line tool

pyxform is used as an underlying library in several form-conversion tools.

For some people, the command-line tool can be convenient and it's easy to install with pip install pyxform.

https://github.com/XLSForm/pyxform

Downloadable files

Where (and whether) to put them in the docs.
And then document how to link to them.

Calculates are evaluated even when not relevant

https://forum.opendatakit.org/t/odk-collect-calculate-fires-even-when-not-relevant/8923 This is somewhat unexpected behavior so it should be explicitly called out.

Replace select-search-start.png with valid PNG

The build is broken because select-search-start.png is not a valid PNG. It should be replaced with one that works.

A brief history of ODK subprojects

Jumping in to opendatakit in order to build a few things for aggregate and collect I've been struggling with the context of everything.

A few things are in a state of ownership flux, moving (to my understanding) from University of Washington 'ownership' to community driven. A lot of this knowledge is not easy to come by for newcomers - a prime example is Javarosa*, which has one retired-ish version on bitbucket and a recent version on github under odk's wing.

A brief history and current state/status of these project changes would help provide some context when reasoning about how things all fit in together. Leading to less surprises and pointers on where to find the 'current' information on things.

* : so, e.g with javarosa I had an issue with digging up the right version of it to get aggregate building, had to jump through things referencing each other and get help to figure out what was going on.

Create a system diagram for the website

Originally reported on Google Code with ID 594

1) Create a new system diagram
2) Update the research page on wiki

Reported by wbrunette on 2012-06-06 18:14:11

Android-level security recommendations

For example, if enumerators are able to install apps, they can install camera apps that may let them bypass certain requirements (see here).

Another example -- turning on Android-level data encryption means when the device is locked, no one can see the data. It's an easier alternative to using encrypted forms that offers most of the benefits.

Add and document the javasphinx java domain

(Also, make sure it even works.)

https://bronto.github.io/javasphinx/

Move binary files to GLFS

We currently store images and videos directly inside the docs repo. This causes the size of the repo to grow quickly and makes for slow cloning and fetching.

I think we should move these files to Git LFS so we can store binary files at a high resolution, but be able to pull and compress them (e.g., #64) at publication time.

Using offline map tiles

This isn't currently documented anywhere.

getodk/getodk#1260 is the original issue filed by @florianm and includes a link to a basic description here.

Document Aggregate limitations

Some good ones on this issue - getodk/briefcase#142 (comment) and I'm sure there are more on the dev wiki

Mention expected 400 errors from "Upload ODK Aggregate"

Step 17 of the instructions for installing Aggregate on App Engine (Cloud) reads:

17. Now click Upload ODK Aggregate.
    This will spew a very long list of progress messages into the Output window. The listBackends : and deleteBackendBackground : sections may report "500 Internal Server Error" and Severe errors, and Warnings about the use of Backends, a deprecated feature. This is expected.

Toward the bottom, the update : section should not report errors and at the end, a status : Action Succeeded! line should be written. This indicates that the upload completed successfully.

An example of the scrolling progress messages produced by a successful upload is here.

I thought it was helpful that the instructions list the errors you are expected to encounter, for example, "500 Internal Server Error". I encountered several "400 Bad Request" errors, which I initially thought indicated a problem with my installation, but after looking at the example log mentioned in the instructions, it looks like those are normal. I'd recommend explicitly mentioning those 400 errors in the instructions so that it's clear they're expected.

Include OpenRosa documentation

https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaAPI -- some of this is obsolete but most of it is still applicable to ODK JavaRosa which powers Collect. It also describes the interaction between servers and clients in the ecosystem. This is only relevant to developers.

The form specification has already been pulled out to https://github.com/opendatakit/xforms-spec and is being maintained there. Should it continue to live separately or be incorporated into what these docs will become?

Describe how loading external settings works

getodk/collect#456

Bulk configuration through QR codes

getodk/collect#781

https://opendatakit.org/2017/06/configure-collect-on-many-devices-with-qr-codes/

Collect form widgets, with XLSForm and XForm details

part of #20

essentially this: https://opendatakit.org/help/form-design/examples/
but:

• add XLSForm details
• include newer widgets not currently documented
• better screenshots (?)

Seperate docs folder for each ODK tool

@adammichaelwood I propose that we have a folder for each ODK tool/component and keep the docs for the specific tool in that folder. Most if not all tools might have the same structure and it will be very difficult to name the files with the current setup. For example we already have a getting-started.rst (for Collect), we can not have another getting-started.rst say for ODK build or ODK Aggregate with the current set up.

License in docs

CC BY 4.0, to display in footer of docs output

Briefcase "use parallel connections" feature and when it might be helpful/harmful

getodk/briefcase#142

Contributor Guide

We need to come up with guides. I am thinking of:

1. Contributing Guide
   example:
   https://github.com/rackerlabs/rackspace-how-to/blob/master/CONTRIBUTING.md

2. Style Guide
   examples:
   https://github.com/rackerlabs/rackspace-how-to/blob/master/style-guidelines.md
   http://help.apple.com/asg/mac/2017/
   https://developers.google.com/style/

Blank values sent to gsheets result in cell with a space

See https://forum.opendatakit.org/t/space-written-to-empty-sheets-cells/7002

As mentioned, downstream analysis tools can trim cell values or structure their empty test as "either the cell is empty or includes a single space."

Add "Was this helpful?" or other feedback mechanism on docs page

add from @ronna comment on #55:

We can also have a "Was this information helpful?" at the bottom of the page, with an option to send feedback
example here (right at the bottom of page)

Differences between Aggregate and Briefcase

The practical difference between Aggregate and Briefcase is often clear — for example, many features are available in one but not the other — but I've struggled with the conceptual difference. Where does Aggregate end and Briefcase begin? For example, when do you export data from Aggregate vs. export from Briefcase?

I discussed with Yaw and Hélène, who described the following differences and suggested I create an issue in the docs repo:

• Briefcase is good for offline use, for when there's no need for a server. Briefcase is often enough for smaller projects. However, note that Aggregate can also be used offline and locally.
• Briefcase can be used to back up all forms and submissions on Aggregate. In particular, Briefcase's command line interface makes this easier.
• Briefcase is needed to transfer data between Aggregate installs.
• Briefcase is required to decrypt any encrypted submissions.

Add Enketo screen shots to Widget guide

work on a complete widget guide is in progress, but first pass is only ODK Collect screen shots. Add Enketo screen shots would be good too.

Add a LICENSE.md to the docs repo

What we've historically used (e.g., for governance) is Attribution-ShareAlike 4.0 International (CC BY-SA 4.0), but I don't think we want to force virality. Our code is Apache 2 and our docs should be Apache 2 like. For that reason, I think the license should be Attribution 4.0 International (CC BY 4.0)

Describe all intents

@rishsethia put together some docs about using Collect's Intents at getodk/collect#74 that need a place to be!

Pull in docs from community sources

For each of these, we need to first ask permission!

• https://www.google.com/earth/outreach/learn
• http://sudan.validmeasures.org/category/open-data-kit
• http://opendatakit.lshtm.ac.uk/what-do-i-need
• http://odk.acf-e.org/odk/start_here.html
• https://opendatakit.org/help/training-guides (lots o links here)

Describe how to use lossless compression on source images

In the contribution guide, we should describe how to use ImageOptim (or pngout for pngs and mozjpeg for jpgs) to do lossless compression on images.

Reduce the size of PNGs before publishing

Currently, the images in the repo are large (as far as bits). For example, _images/annotate-1.png is 1.7 MB, but running it through ImageOptim at 80% compression brings it down to 595 KB. Reducing the size of the images makes for much faster loading (important in the areas where ODK is used) and it also saves us money.

An ideal solution to this problem would compress the images as part of the sphinx build process.

Setup CircleCI to publish to S3

• Commit from @adammichaelwood with command to build locally
• Add circle.yml file to build master and pull requests
• Add badge to readme.md to indicate build status
• Add reporting to Slack #docs-pulse for failed builds
• Setup S3 bucket to hold docs
• Setup DNS, Cloudfront for https://docs.opendatakit.org
• Add keys to CircleCI to automatically upload docs to bucket
• Setup SSL for https://docs.opendatakit.org
• Add command to upload builds to S3
• Verify builds trigger CloudFront updates