muckrock/api-examples

A collection of scripts using the MuckRock API.

MuckRock API

This repository presents a collection of scripts demonstrating the MuckRock API. Find more information on filing requests in bulk on our Notion guide.

The API has a visual interface, but it is most powerful when accessed programmatically.

To access the API, you must first get your API token, which is currently available on your profile page (scroll to the bottom left). The key is generated from your MuckRock user account and password. If you don't have an account, sign up here.

api-examples's Issues

FYI, export_stats.py doesn't work as expected

Just tried to run it on user GeoCen, got back a CSV file with only header rows and nothing else (http://cl.ly/2l0b1p0t2k1D).

python code is broken

utils.py:19: response = requests.post('https://www.muckrock.com/api_v1/token-auth/', data={'username': user, 'password': pw})

https://www.muckrock.com/api_v1/token-auth/ is a 404 now and seems to be missing from the API entirely based on the api root page

Need example using the API to find traffic stop data files for a specific agency

Is there an example using the API to find traffic stop data files for a specific agency? For example, the Pittsburgh Bureau of Police files

I can use the API and specify the agency and tags with "traffic enforcement"

https://www.muckrock.com/api_v1/foia/?format=json&jurisdiction=&agency=357&projects=&tags=traffic+enforcement&has_datetime_submitted=true&has_datetime_done=true&ordering=datetime_done

and return a collection of communications to look through for a "ffile" that may be data. However, there doesn't seem to be metadata associated with the "ffile" to define it as traffic stop data. Also it is not clear if the "traffic enforcement" tags are standardized.

If the example does not exist, I would be happy to write one and submit a PR, but it doesn't appear the metadata is available that defines the information in the "ffile". Is that correct? Thanks!

lots of errors when I try to export files

I'm trying to export all requests for user GeoCen into files/folders in the way that Shawn has provided them to me in the past (cf http://cl.ly/image/3B0Z3x1d3z2m) so I don't have to bother him! I downloaded the repo and tried to run some of the scripts but am getting weird undocumented errors. e.g.

chris$ python export_users_requests_as_files.py 
Username: GeoCen
Password: 
Username to export (case sensitive):GeoCen
URL I'm using is https://www.muckrock.com/api_v1/foia/?user=GeoCen
hey rob. check out <Response [500]>
There was an error of unkown origin
URL I'm using is https://www.muckrock.com/api_v1/foia/?user=GeoCen
hey rob. check out <Response [500]>
There was an error of unkown origin

Here's the traceback when I interrupt that process: http://pastebin.com/S7yRkbh3

Or this one:

chris$ python export_all_user_requests.py 
Username: GeoCen
Password: 
Username: GeoCen
Traceback (most recent call last):
  File "export_all_user_requests.py", line 19, in <module>
    data = r.json()
  File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 819, in json
    return json.loads(self.text, **kwargs)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Or this one:

chris$ python export_as_files.py 
Username: GeoCen
Password: 
Working on request 6996
Traceback (most recent call last):
  File "export_as_files.py", line 19, in <module>
    request_data = request.json()
  File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 819, in json
    return json.loads(self.text, **kwargs)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Am I fundamentally misunderstanding how these ought to work? Is there something broken w/ my configuration/python? Or is this broken on the API end?

Significant amount of public information accessible over API

The API exposes a large amount of information including addresses, phone numbers, first/last names, etc. While much of this information is public record by being sent along with requests, many of those individuals may have privacy concerns because of the ease of access. Particularly for those who don't know that this is easily possible. This is a reservation that prevents me from going through Muckrock for >100 requests.

Error of unknown origin while trying to use export_users_requests_as_files.py

Trying to export a bunch of data about my requests but getting getting an unknown error.

API-examples-master chris$ python export_users_requests_as_files.py 
Username: GeoCen
Password: 
Username to export (case sensitive):GeoCen
Page 1 of 18
Working on request 7540
Here is the agency number 2822
There was an error of unkown origin
Page 1 of 18
Working on request 7755
Here is the agency number 2682
There was an error of unkown origin
Page 1 of 18
Working on request 7748
Here is the agency number 2675
^CThere was an error of unkown origin
Page 1 of 18
Working on request 7660
Here is the agency number 2948
There was an error of unkown origin

New example: Download a request

Until we provide the functionality on MuckRock to one-click download a request, an example script for downloading all the files and communications into a local folder would be very helpful.

add scripts to export MuckRock IDs & info to JSON

Pull #10