uscensusbureau / api Goto Github PK
View Code? Open in Web Editor NEWempty repository solely used for API related issue tracking, bug reports and feature requests
Home Page: https://census.gov/developers
empty repository solely used for API related issue tracking, bug reports and feature requests
Home Page: https://census.gov/developers
Hi @loganpowell -
A tidycensus user just alerted me to an issue with block group IDs coming out of the 2013-2017 5-year ACS API over in walkerke/tidycensus#347. block group
is returning null
when requested from the API - see here for an example. I've run through a few states/counties to test this out and it is consistently missing. This does not appear to be an issue for any other year. Thanks for looking into this when you get the chance!
Kyle
To specify the geographic summary level requested, the various API endpoints take a for
URL parameter with english language strings as the acceptable values.
It would be good if the summary level codes were usable in addition to strings, e.g. 310
for metropolitan statistical area/micropolitan statistical area
repost from Slack
Should top-coded variables be mentioned in Notes on ACS Estimate and Annotation Values ?
I just realized some differences between API returns and files download from data.census.gov in terms of how they store top-coded values.
For median household income variable in Table B19013, there are two tracts in New York County are coded as 250,000+ from the csv file I downloaded via data.census.gov , but in the API return, it is coded as 250,001. I wonder if it is a good idea to remind folks about this special case in the Notes since most of the time when I send API requests, I barely request annotations.
Hi,
Thanks for all the work you do with the census api. I'm using tidycensus
to download dissertation data for all tracts in the country for a number of variables. I've run the below call several times over the past few days to get 2000 decennial data, but I keep getting errors. ( I'm hone my script by just calling data for CT and 75% of the variables I need). I've contacted the author of the tidycensus
package, but we don't think the error is with the package.
# assign selected variables
my_vars00sf1 <- c(pop = "P001001",
# race-ethnicity
racetot = "P007001", nhwht = "P007002", nhblk = "P007003", nhnat = "P007004", nhasian = "P007005",
nhpac = "P007006", nhother = "P007007", nhtwo = "P007008", hisptot = "P008001", nothisp = "P008002",
hisp = "P008010",
# sex
sex_tot = "P012001", sex_m = "P012002", sex_f = "P012026",
# median age
mage = "P013001", mage_m = "P013002", mage_f = "P013003",
# housing unit
hu = "H003001", occ = "H003002", vac = "H003003", occ_tot = "H004001", owner = "H004002",
renter = "H004003", hhsize = "H012001", hhsizeo = "H012002", hhsizer = "H012003")
# run function to call all sf1 for country
sf1 <- get_decennial(state = "CT",
geography = "tract", # specify tract geography
year = 2000, # specify year, 2000
variables = my_vars00sf1 , # set variables
sumfile = "sf1", # specify sum file for decennial
show_call = TRUE, # show call for troubleshooting
geometry = FALSE) # specify that we want the output to be wide
# glimpse to verify call worked
head(sf1)
Error in load_data_decennial(geography, variables, key, year, sumfile = "sf3", :
The Census API has returned the error message error: error: unknown variable 'H003002'.
This may be due to mixing SF1 and SF3 variables. If so, separate your requests to SF1 and SF3 when using `get_decennial()` by using the `sumfile` argument in separate calls.
Error in gather.default(., key = variable, value = value, -GEOID, -NAME) :
object 'NAME' not found
I know there was some issues over the last few months with new endpoints, so perhaps some issue remain. I can run the call with different combinations of variables and sometimes it is successful and sometimes not. "H003002" is a problematic variable that seems to be tripping up the call. Just calling H003002 by itself usually works, but whenever it is included in a call with more than two other vars, it seems to break the call.
Is anyone else having issues with this or is there something I'm missing here? I'm also having similar issues with SF3 calls too. Any help would be greatly appreciated.
Thanks.
Census APIs which return data as JSON should take advantage of JSON's data typing. Numeric values should not be served with quotes. Take this API call, for instance:
As can be seen in this screenshot of a browser JSON viewer:
all values are quoted, and thus interpreted by JSON parsers as strings. However, the API documentation shows clearly that the predicateType
for most of these variables is int
.
By returning the numeric values without quotes, most JSON tools would parse the API response and return those as numeric values, saving API users from redundant work and making the real nature of the data more clear.
It would be incredibly useful if earlier population estimates were added to the population estimates APIs, https://www.census.gov/data/developers/data-sets/popest-popproj/popest.html.
State estimates are only available 2000+. The 1990s intercensals only have counties broken down by subgroups, no totals or state totals. For earlier data I could only find a bunch of different txt and csv files that aren’t really formatted consistently year to year. If there’s ever any time to go through the backlog adding in older intercensals would be great.
It's very tedious that you have to get the census data (e.g., ACS) via Census API and then find a shapefile to download through TIGER and merge them. Usually you need to merge them based on their FIPS code, but it sometimes changes from year to year. Let's say the Census decides to add a few more tracts in a county or change their area. In other words, you have to download, for example, ACS 2015 shapefile to match with ACS 2015 data. This is very frustrating and also error-prone as some people don't realize this.
I think a very simple solution would be using WKT to store the geometry info and adding a field in the JSON file returned so that users can covert it to a geo-file after the request from their end.
https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry
I know that the PUMS API is not "official" yet; however I did notice that Census is starting to advertise the data.census.gov Microdata tool (e.g. http://apdu.org/2020/05/27/new-microdata-access-tool-for-acs-and-cps/), which has the API as one of the output options. Right now, household serial numbers for the 2018 ACS do not come through correctly, e.g. at this example call to the API:
[["SERIALNO","SPORDER","WGTP","PWGTP","AGEP","PUMA","ST"],
["*","1","42","43","58","100","50"],
["*","4","43","16","14","200","50"],
["*","3","43","37","15","200","50"],
["*","2","43","23","42","200","50"],
["*","1","43","43","49","200","50"],
["*","2","58","34","10","300","50"],
...
It would be fantastic to get household serial numbers to come through the API for 2018 correctly as this would enable use of replicate weights, appropriate variance estimation in modeling, etc. I haven't noticed this issue for earlier years. Thanks for all your hard work on this, I appreciate it!
Hi @loganpowell -
I was fixing some bugs over in tidycensus today and I noticed that block group data appear to have been removed from the ACS APIs for the 2008-2012 5-year dataset and earlier. Block group is also no longer listed as an available geography over on the website: https://api.census.gov/data/2012/acs/acs5/geography.html.
Do you know if this is a temporary removal or is it permanent? Thanks!
Data returned from data.census.gov and api.census.gov can include a GEO_ID
value which is a key for linking data across queries, as well as linking to GIS data.
However, the structure of some of these GEO_IDs is obscure, and should be documented more clearly. There is a page, Understanding Geographic Identifiers (GEOIDs), which is a good start. However, the only mention it makes of the "geographic variant" is pointing out that it is 00
in an example GEO_ID.
However, for several geographies, the GEO_ID values returned have something other than 00
in that position.
M#
with different numeric values for #
L#
with different numeric values for #
U#
with different numeric values for #
There may be more; these are the ones I've found.
Whether in the API or some other source, there ought to be a complete enumeration of possible geographic variants with reference information about their sources. At a minimum, a reference from a given variant to a given TIGER vintage should be documented.
For CBSA/CSAs, which are formally delineated as lists of one or more counties, it would be valuable to be able to link the variant to a specific delineation. Experimentally, I've worked out that M5
is the Sep. 2018 delineation, and M4
is the Apr. 2018 delineation, but it ought to be systematically published.
For the State Legislative Districts, the API returns names which include a year in parentheses, and one can infer that L6
and U6
are the 2018 legislative year, and L5
and U5
are the 2016 legislative year, but, again, this ought to be made explicit.
Here's an example.
Table: B11002
has a structure like the following and you have to see it via the Census' interface
It is really hard to see this structure and level if you only download data via API. You have to go over a long table and each level is separated with !!
https://api.census.gov/data/2018/acs/acs5/groups/B11002.html
It is also really error-prone and I can't recall how many times I've seen people get white population when attempting to get the non-Hispanic white.
It would be great if there is an API to return a tree structure that resembles what users see via the interface. For example:
Estimate
├── Total -- B11002_001E
└── Total
├── In family households -- B11002_002E
├── In family households
│ ├── In married-couple family -- B11002_003E
│ ├── In married-couple family
│ │ ├── Relatives -- B11002_004E
│ │ └── Nonrelatives -- B11002_005E
│ ├── In male householder, no wife present, family -- B11002_006E
│ ├── In male householder, no wife present, family
│ │ ├── Relatives -- B11002_007E
│ │ └── Nonrelatives -- B11002_008E
│ ├── In female householder, no husband present, family -- B11002_009E
│ └── In female householder, no husband present, family
│ ├── Relatives -- B11002_010E
│ └── Nonrelatives -- B11002_011E
└── In nonfamily households -- B11002_012E
Referencing walkerke/tidycensus#424, we see that the old API endpoint for the 2005-2009 ACS still exists at https://api.census.gov/data/2009/acs5, and contains some variables that are not available at the newer endpoint https://api.census.gov/data/2009/acs/acs5. For example:
https://api.census.gov/data/2009/acs5?get=B08101_001E,NAME&for=tract&in=state:01
works,
but https://api.census.gov/data/2009/acs/acs5?get=NAME,B08101_001E&for=state does not.
Posting here to put on your radar for after the holiday. Thanks!
Some variables in some Tables in ACS only available for some geo-level. It would be nice if you guys could add a new field to the following json to indicate all the available geo-levels.
https://api.census.gov/data/2018/acs/acs5/variables/B01001A_001E.json
{
"name": "B01001A_001E",
"label": "Estimate!!Total",
"concept": "SEX BY AGE (WHITE ALONE)",
"predicateType": "int",
"group": "B01001A",
"limit": 0,
"attributes": "B01001A_001EA,B01001A_001M,B01001A_001MA"
}
While the ACS 5-year is often described as having "data for all geographies down to block group," there are a number of tables for which data are not available for all geographic levels.
This is laid out in detail in a spreadsheet. It would be great to be able to get these restrictions as part of the group metadata.
A minimum would be to simply add the text from column C as a field in the per-group metadata; my preference would be an explicit list of summary levels (codes and/or text identifiers as used in the API) that are either available or not available.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.