associatedpress/harvester

Collaborative data collection tool developed by the Associated Press

harvester's Issues

Docs should recommend creating a new GCP project?

AFAICT a GCP project can only have one Oauth consent form. If another app is configured to use Oauth in a user's sole GCP project, it means Harvester would need to share that same form. This appears to be possible (i.e. you can add multiple Redirect URIs and Authorized domains), but I believe they would need to share the same user verification screen and related metadata about user policy, etc.

Might be a good idea to instruct users to create a new GCP project specifically for Harvester.

Google credential documentation additions (OAuth, consent screen)

Some Google credential additions suggested by @zstumgoren

1. Explain why users are setting up OAuth (again) in https://harvester.readthedocs.io/en/latest/google_credentials/#oauth-authentication E.g.

   We recommend setting up your Harvester instance so that users must log in before they can add data. This step will set up the application in way that anyone who you’ve shared the Google Sheet with will be able to log into Harvester and perform data entry.

2. Explain what a Consent screen actually is here https://harvester.readthedocs.io/en/latest/google_credentials/#consent-screen E.g.

   A consent screen is a web form that users will see when they attempt to log into a form on the Harvester site. This screen will ask users to choose a gmail account with which to log in (if they have multiple accounts, they should choose the one that the project’s Sheet was shared with).

Add use cases to documentation

A common question I'm seeing is "how is this tool useful?"
A related question is "why would one use this rather than Google Forms?"
I think both could be answered with some vignettes of common use cases with particular
focus on the features Google Forms didn't accomplish for us.

Allow deep linking to specific entries

It's great that we're able to go to a Harvester form that has an index configured, fill out the search form and pull up the latest entries (or a blank form) for the specified index—but for ease of communication, it'd also be useful to be able to link directly to that entry as needed.

Put differently: Rather than give someone (e.g., by email) the instructions to go to https://harvester.ap.org/d/1Bhmk_cLtqd7VjOUnnfHEeFltztZ_yCuisdrYFctCLFs and search for Sir Launcelot of Camelot, can we just send a link to https://harvester.ap.org/d/1Bhmk_cLtqd7VjOUnnfHEeFltztZ_yCuisdrYFctCLFs#index=Sir+Launcelot+of+Camelot?

Custom Form URLs docs should say form_id not doc_id

Docs for Custom Form URLs should replace doc_id with form_id:

"The app configuration sheet allows you to set up custom form URLs in order to
get a more human-readable link to share with data collectors. If the config
sheet has a tab named forms that contains the columns slug and doc_id form_id
then you can access the form specified by the sheet with ID doc_id form_id at the URL"

Compile guidance on creating a questionnaire

In some instances the data that needs to be collected using Harvester will be obvious, but in others, identifying the form and format of what should be collected may be more challenging. Coming up with a questionnaire that makes sure the data collected will be useful for data analysis may benefit from research on data collection and survey best practices.

Below are some links that may help:

• https://medium.com/knowledgenudge/how-to-create-effective-research-surveys-c282f134beb3
• https://www.pewresearch.org/methods/u-s-survey-research/questionnaire-design/

Clarify usage inside a company GSuite

Usage of Harvester seems to get a little more difficult if you use a company GSuite to manage your Google sheets and whatnot. Even if you make a sheet globally writable it seems like it won't be accessible to users not inside your GSuite. You would probably need to create the Harvester service account inside your GSuite. We should clarify that in the documentation.

Docs should mention need to enable Drive API on GCP project

As part of the Google Cloud Platform setup process, it's necessary to enable the Drive API for the GCP project that will be used for Harvester (i.e. the project where Oauth credentials and the new service account will be created). Probably worth mentioning this on the setup docs.

If Drive API is not enabled, application loading fails with the below error (from Heroku app logs):

2021-03-11T21:12:19.011715+00:00 app[web.1]: (node:31) UnhandledPromiseRejectionWarning: Error: Access Not Configured. Drive API has not been used in project XXXXXXXXXX before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/drive.googleapis.com/overview?project=XXXXXXXXXX then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

"Example" text for string fields

Currently, string fields are blank in the form. Idea: define example:"example text" which then populates the string form with "example text" in slightly grayed out text (similar to how "Select..." renders on select fields) until the user fills it in.

Docs should mention need to enable Sheets API on GCP project

Related to #4 #6

As part of the Google Cloud Platform setup process, it appears to be necessary to enable the Sheets API for the GCP project that will be used for Harvester (i.e. the project where Oauth credentials and the new service account will be created). Probably worth mentioning this on the setup docs.

If Sheets API is not enabled, attempting to load a Harvester form produces the below error (from Heroku app logs):

message: 'Google Sheets API has not been used in project XXXXXXXXX before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/sheets.googleapis.com/overview?project=XXXXXXXXX then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.',

Docs should mention need to enable People API

Related #4

Looks like the scopes mentioned on setup docs require enabling the People API for the GCP project.

If this API is not enabled, login fails on the root Heroku domain and the following error is generated in Heroku logs:

<<< snipped >>>

2021-03-12T02:14:38.024833+00:00 app[web.1]: GaxiosError: People API has not been used in project XXXXXXXXX before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/people.googleapis.com/overview?project=XXXXXXXXX then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

<<< snipped >>>