18f / api.data.gov Goto Github PK
View Code? Open in Web Editor NEWA hosted, shared-service that provides an API key, analytics, and proxy solution for government web services.
Home Page: https://api.data.gov
License: Other
A hosted, shared-service that provides an API key, analytics, and proxy solution for government web services.
Home Page: https://api.data.gov
License: Other
The edit links at the bottom of pages seem to go to the master branch of this repo, but is it that or the gh-pages branch that they'd want to edit in order to suggest an edit to the site?
If gh-pages, would it make sense to possibly remove the master branch?
Right now the key signup process is: fill out the signup form, immediately get a key. If you want a new key, you fill out the form again.
There's been periodic discussion over creating more of an "account" concept, so users might have a password to login and view and manage multiple API keys they might have for different projects. This could also be a place for users to view analytics on their own API key usage. We've also discussed using MyUSA for this account login mechanism.
I think we have some parties that may still want the no-realy-signup process for getting api keys, but I think others are interested in more of a required signup process. So maybe make it optional? For example, fill out your name and e-mail to get a key, or signup for an account to manage your keys, view analytics, etc? I'm not exactly sure how all this might work, or what we really want to do, so this would obviously require a bit more discussion.
Hi there, I recently stumbled upon api.data.gov and applaud all the awesome initiatives like this one happening right now - thank you!
One confusion I think should be address on this site is that even though it is a subdomain of data.gov, it isnt actually an API of data.gov whose APIs are documented at http://www.data.gov/developers/apis.
Two potential solutions:
http://api.data.gov/community/
I found that page while googling and can't tell how it exists (I don't see that in the gh-pages or master branch). Any thoughts on that? Any other pages like this exist?
If possible, it'd be good to axe that page entirely so it's not left out there like a shell.
Any reason for us not to move the data.gov link on the homepage to the GSA page? It's a gsa site and I'm inclined to keep that consistent.
Should the issue tracker for this repo just be for the project website at api.data.gov or should it also be the issue tracker for the service itself? Or, would the latter be better suited elsewhere, like here?
It would be nice to have a few chart and tables available for the administrative dashboard. Maybe start simple with an activity graph of the last few hours, already filtered to certain API's.
If an agency is a user of api.data.gov, we want to offer them basic code samples they can use for embedding the following within their website or elsewhere:
API.Data.gov v. api.data.gov
The HTTP caching layer of api.data.gov is currently disabled. We need to get it re-enabed, but we need to validate some things first.
The caching layer got disabled when we encountered this bug with OPTIONS requests. We've worked around the issue on our end, and we've also got the underlying issue patched in NodeJS. However, while this was disabled, I discovered that some of our unrelated Varnish caching servers have been experiencing periodic failed requests due to this gzip+chunked responses+streaming issue in Varnish. The issue is very sporadic and seems to affect a very small percentage of requests, but it's still concerning, and I'd like to sort it out before re-enabling caching. I'm not 100% sure this same problem is happening on our api.data.gov servers, so that needs to be verified, but I suspect it might be if Varnish gets re-enabled (since we saw this issue on a few other different Varnish setups where streaming was enabled).
So before re-enabling caching:
Add a v. brief description for these repos:
The main graph of API traffic in the admin can be switched between hourly/weekly/monthly views. We've actually already implemented a minutely mode, but it's currently hidden for performance reasons (it can be a lot of data to pump back if you happen to select minutely while looking at a couple months worth of data).
This minutely view can be very helpful for diagnosing traffic spikes, and regulations.gov has been making use of this. We should do a couple little things to make this more usable for everyone:
Noticing the example of www.regulations.gov/developers, where they link offsite to the api documentation on api.data.gov. We want to ensure that they could easily integrate any of these parts directly into their site.
roadmap
loose notes
We've had several requests from API users who would like to view usage metrics on their own API keys. This might be similar to the analytics we give to admins, but given to users to view on their own API keys.
perhaps a faq or section of /about.
The goal would be to have just 1 sentence explaining what each of the repos are and how they interact.
We've had two reports of the api key emails getting caught in a spam filter. I'm not sure whether those are flukes but it's worth keeping an eye on and possibly testing more around.
In the analytics admin, you can select a range of dates to perform your analytics queries on. The date range picker also includes some common ranges like "last 7 days" or "this month". If I select the last 7 days on 2014-07-21 then I'll end up with a URL like:
/admin/#/stats/users/tz=America%2FDenver&start=2014-07-15&end=2014-07-21&interval=day
However, if I bookmark that URL or share it with other admins, they will always be viewing this 7/15-7/21 time frame. This static view of dates may be desirable in some circumstances, but I think if I've selected the "last 7 days" option, then most of the time I want a URL that will always show the last 7 days, regardless of when I access. This would allow you to bookmark that URL or shared URLs for analytics queries you regularly have to pull (for example, if you have to pull your active users every month).
My general thoughts of how this might work:
All of this really builds towards being able to save common reports, which has been on the radar for a while: NREL/api-umbrella#16 But perhaps just by making the URLs bookmarkable with relative date ranges, that accomplishes most of that goal.
@shawnjohnson Thanks for the idea to use elasticsearch's date math! I hadn't seen that before, but that seems like it'll definitely make this easier and more flexible to implement. Otherwise, does what I've outlined here seem logical to you? Mainly that the date range picker will always result in relative URLs unless you pick "Custom Range" and explicitly pick dates? And while I think exposing the date math in the URL will give you whatever flexibility you need, I am curious what relative date ranges you commonly run queries for, since I'd like to add those to the date range picker. I commonly have to run reports based on fiscal quarters, so I'll probably be adding those ranges, but if you have other frequent ranges, those would be great to know.
Shooting to schedule during the August visit
FBOpen is rapidly heading in a direction where we need to have a write API. There are other 18F projects (cc @dwcaraway) which will need this capability, too.
Stating the obvious, we will also need a way to control roles and permissions for API keys, since we'll need to limit write access to certain subsets of our users.
When an administrator logs into the admin part of api.data.gov, they should only be able to view their own agencies APIs configuration, analytics, etc. They should then also have the ability to publish their own API backend configuration without publishing any other pending configuration changes from other agencies.
This ticket has more of the technical details: NREL/api-umbrella#9
http://api.data.gov/contact needs updating theme-wise.
But I also wonder if it's better just to point people to the issue tracker? Up to you, though.
It was reported that our swagger docs on api.data.gov aren't working in Internet Explorer. Need to verify which versions of IE this is affecting and figure out if this is something we're doing wrong or a bug on Swagger's end.
One of our users reported that they new a client made ~194,000 requests on July 22. However, in our view of the analytics, only 24,652 requests were made. Need to investigate what's up. The account ID that reportedly made the requests is 758f2ff8-434e-4d7e-b1e0-af15d4a94f26
.
As we recently saw with the Regulations.gov API, wee need to provide a means for the API owner to post information on api.data.gov about upcoming downtime.
In this case, the regulations.gov team had posted a notice on the API homepage on their website, but users who only browsed api.data.gov wouldn't have known.
My suggestion would be to have a simple alert box built into each documentation page (perhaps above the description or floating on the right), as well as an alert box on the api.data.gov homepage. Have it built with placeholder content, then commented out. Then add info to the about page that tells agencies about it and how to employ it via pull request, and be sure to alert existing and new users about it and the importance that they use it.
We're beginning to get recurring requests for certain data points, namely:
To align this project with GSA and 18F's source code policy, we'd like to move the content in this repo into the public domain (this is mainly the documentation and static web page content on api.data.gov). Right now this repo is missing a license altogether, so it would also be good to make this explicit.
To previous contributors to this project, @shawnjohnson, @nvembar, @brookeheaton: Can you confirm with a ๐ if you are okay with relinquishing all copyright claims to your prior contributions to the project, both code and text, and moving this project to the terms of CC0 Public Domain Dedication? If you're not okay with this, or have any questions, just let us know. Thanks!
HTTPS is already configured at https://api.data.gov. So, api.data.gov should make it the recommended and default protocol.
I'm sure the team here already understands the various benefits of HTTPS, and I don't mean to preach to the choir, but here are a few:
Much of the performance hit involved in SSL can be mitigated - @igrigorik's excellent istlsfastyet.com contains many references for doing this.
There are a few ways this issue could be tackled:
https://
in the examples.https://
.http://
requests to https://
for the main api.data.gov
website.http://
requests to https://
for any contained API.https://
going forward.The MyUSA login for the administrative side of api.data.gov is currently not working. Need to investigate and look into upgrading the omniauth-myusa gem.
this repo, that is.
Data.gov's redesign is fully launched and has been rolled out to the rest of it's subdomains, so we should do the same for api.data.gov.
@GUI - let me know what would make that easy and we can try to line things up so as to keep this a minimal effort.
This will address the Data.gov issue #390
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.