gsa/code-gov-data

Data Files used by code.gov

code-gov-data

Data used by Code.gov

Setup instructions

To run this app locally you'll need a Code.gov API key. If you don't have one, go to the https://developers.code.gov/key.html to obtain one.

Once you have an API key, clone the repo and run the npm install command in the project's root directory to install all of the project’s dependencies.

Run cp env.example .env to create a template .env file in the root directory of the project

Replace “[your api key goes here]” in the .env file with your API key

Filters

This repository is used to generate the repos.json and tasks.json files that populate the filters on the browse projects page and the open tasks page of ‘code-gov-front-end’.

Generating new filters

Create a feature branch using Code.gov naming convention (e.g., jrc-update-filters). If there are no filter updates to commit and merge with master, the feature branch can be deleted.

Run the npm run generate terminal command which runs 2 commands:

• `npm run build-filters’ creates the filters/repos folder with the filters of the browse projects page
• `npm run build-task-filter-data’ creates the filters/tasks folder with the filters of the open tasks page

Once the script has finished, execute git status to see if any of the data changed. If so, commit the changes and submit a PR to master. These changes will not be reflected on Code.gov until a new release of code.gov is deployed following the instructions in the Front End Release Management wiki page of code-gov-front-end.

name_with_owner.txt [DEPRECATED]

This text file includes all of the names of the Open Source repos on Code.gov. It is generated by extracting the name_with_owner column from code-gov-gh-repos-stats.csv

Generated by running awk -F "\"*,\"*" 'NR>=2 { print $4 }' code-gov-gh-repos-stats.csv > name_with_owner.txt

code-gov-data's Issues

"><img src=x onerror=prompt(123)>

">

Add documentation

Document the scripts that are used in this repo. What are they/what do they do/steps to update this on the front end.

Intended date/time format for schema

https://github.com/GSA/code-gov-web/blob/master/src/assets/schemas/2.0.0.json#L243 has comments of "in YYYY-MM-DD or ISO 8601 format" for the dates. Is the intent to allow both date and date time formats or just a date format? The JSON Schema spec at http://json-schema.org/latest/json-schema-validation.html#rfc.section.7.3.1 shows "date" as being the same thing as "full-date" (date is same thing as full-date according to https://tools.ietf.org/html/rfc3339#section-5.6 ). Neither includes time. The current "date" value specified in the code.gov schema means just a date without time which explains why I'm getting errors with the library I'm using (https://github.com/JamesNK/Newtonsoft.Json.Schema/).

I'm taking dates from GitHub API calls and assigning them to certain code.gov schema fields but the schema isn't validating. Seems like the GitHub API uses the equivalent to date-time as far JSON Schema spec and RFC3339 is concerned.

Fix would be to either change code.gov schema to use date-time or I'd change my code to output a date string that's valid with what's currently in the schema (date). Former would probably be too big of a change than the later, but just wanted to ask what the intent was (date versus date-time). Seems like the intent is that code.gov schema wants date without time.

Add contributors to code.json schema?

In reviewing the code.json schema, I don't see a 'contributors' fields, which would be great to include as optional, as it could really put a face(s) on the projects and perhaps compelled folks and humanize the code.

Along those lines, would be great to figure out what fields might give the data more of a sense of activity, action. This could make the projects (and code.gov) feel more 'alive.'

Add additional_fields to schema

Add an additional_fields field to the release object in the schema. This would allow individual agencies to add additional information to their releases.

Extensible Schema

Agencies lead inventory efforts that collect supersets of the information that can be encoded in code.json files. The code.json 2.0.0 schema prohibits adding unanticipated properties. Is there any interest in modifying the code.json schema to make it extensible so that agencies can include their own software inventory fields?

Code.gov Harvester Analytics

Develop Analytics Data from the harvester process to capture the number of exceptions on projects by agency.

List the following columns.

Agency Name with Acronym | # Projects in Agency Code.json File | # of Projects on Code.gov | # of Project Exceptions | Deltas

Save results as: harvester_analytics.csv

Also, detail report showing the projects that are except by agency.

file: harvester_exception_project_list.txt

Permissions Tag Link Broken on Metadata Schema

On https://code.gov/federal-agencies/compliance/inventory-code under "Metdata schema version 2.0.0", when I try to use the "+" drop-down for the "permissions" tag under "array" it directs me toward the bottom of the page, which does not provide any details about what child fields the permissions tag might have.

Please fix so we can know what child fields are under the "permissions" tag of metadata schema version 2.0.0.

Harvester Enhancement - Allow to Run Specific Agency.

Allow harvester to execute the code.json ingestion from specific agencies.