Comments (9)
Does it make sense to add a function to wipe caches? Or is that taken care by forcing the load function to redownload the data. Kind of a power user need.
from cancensus.
The existing use_cache = FALSE
parameter will do just that. I think I mentioned it in the documentation, too.
from cancensus.
(So I think we can close this.)
from cancensus.
I am starting to think that we need to strike a balance between caching and making sure data is up-to-date. For example, on the list_datasets call the user may not find out about new datasets that we might add. Even if it is slow-changing, that might cause problems. Also, the list_vectors will likely undergo changes as I clean up the server side data.
I see three reasons we do caching:
- to avoid unnecessarily spending API points (the reason for the limit on API points is point 2)
- to reduce server load
- to allow people to run and refine their analysis offline
Point 1. only applies to the load_data calls, and I think letting the user decided when to reload the data is good. For the other calls it would be just fine if we only make calls, say, once a day. That the user doesn't have to worry about refinements to the vector data. Or we use the cache-expiry headers from the http call to determine how long they stay fresh, that way we can up that time server side at a later stage when the calls become stable.
from cancensus.
One way to do this would be to cache the caching information from the server (e.g. the ETag) along with the object, and then send an If-Changed-Since
request to the server. I believe this is what browsers do, but I'd have to think about how to do it in R.
from cancensus.
Alternatively, we could store the cache timestamp and force and an update if it gets too old.
from cancensus.
That could work. You could prompt the user with a note that their cached data is old and give them the option to reload, but that might be overkill.
from cancensus.
Actually, it's likely a simple warning would suffice, and be the least intrusive.
from cancensus.
This should be closed now that #54 is merged. If we want to have more discussion of cache invalidation, it should be in a separate issue I think.
from cancensus.
Related Issues (20)
- Bad dependency checks behaviour HOT 6
- Update examples and vignettes for Census 2021 data where appropriate HOT 1
- conflicting namespace HOT 2
- Translation for dataset identifiers HOT 1
- Cross tabulations
- Duplicate vector numbers HOT 2
- Unique CSD names HOT 1
- 1996's DAs calls with province region returns an empty dataframe HOT 2
- WDS endpoint HOT 1
- Original census geographies HOT 2
- Responses get_census() function are just integers divisibles by 5 or "rounded" HOT 2
- Better error message on bulk download requests
- Error when requesting empty regions in 'sf' call.
- [feature request] querying shapes directly HOT 2
- GPT 4 Cancensus HOT 1
- Some CMAs and their provincial sub-parts have the same population values HOT 2
- rgdal and rgeos dependencies HOT 3
- Error: No API key set. HOT 5
- rgdal and rgeos archived
- Broken file paths in get_statcan_geo_suite() HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cancensus.