Comments (6)
Having looked at the code more thoroughly, I think we should:
- Add integration tests for
get_store_from_url
- Add a classmethod
from_url(cls, url: str) -> ""KeyValueStore"
to the definition ofKeyValueStore
. Alternatively, we could add a classmethodfrom_parsed_url(cls, scheme, host, port, path, query, userinfo) -> "KeyValueStore"
, similar toextract_params
. get_store_from_url
then only looks up theschema
and delegates the rest of the logic tofrom_(parsed)_url
- Finally, we mirror what
fsspec
does and allow registering additional stores. Registration would happen through an entry point in the setup.py.
Apart from now being able to register additional stores, this should also clean up _urls.py
, _get_store.py
, _store_creation.py
, and _store_decoration.py
.
FYI: @SimonBohnenQC
from minimalkv.
This concept sounds much better than the current structure 👍🏼
As authentication was an issue in #51, I would suggest testing get_store_from_url
against live AWS / GCS. Can we provide credentials as GitHub secrets to do this @janjagusch?
from minimalkv.
We would probably not be using create_store
, url2dict
, and get_store
internally anymore. We should probably still keep them available, right? Deprecating them seems like the best choice.
from minimalkv.
I would love to keep create_store
as this gives a way to pass parameters where there can also be more schema validation upfront. A typical use case is to have the store configuration in a JSON/YAML and then use create_store(json.load(…))
.
from minimalkv.
I would love to keep
create_store
as this gives a way to pass parameters where there can also be more schema validation upfront. A typical use case is to have the store configuration in a JSON/YAML and then usecreate_store(json.load(…))
.
I see the point of creating stores only from str
or int
parameters. I would also prefer if store objects could be created without relying on an internet connection or successful authentication. Thus, I'd like to move e.g. the create_store_azure
code to a constructor for the AzureBlockBlobStore
class.
What do you mean by "schema validation" in this context?
from minimalkv.
With "schema validation", I mean that you can specify a type schema for JSON/YAML files and validate that they have the correct values before creating the stores themselves. This has been useful in the past in using the stores in some settings to check whether the configuration is valid before deploying it to production.
from minimalkv.
Related Issues (20)
- Boto3 Store is untested code HOT 2
- Handling of colon in Azure KeyValueStores HOT 6
- Ambivalent license HOT 1
- mypy pre-commit Hook HOT 1
- pyupgrade pre-commit hook HOT 1
- Move code from __init__ to (private) submodules HOT 1
- Convert docstrings to numpydoc style
- Restrict possible return types of `_put` and `_delete` methods HOT 4
- `test_git_store` takes approx 15min on 4 cores HOT 1
- Add flake8 to pre-commit hooks
- Add Codecov to CI HOT 2
- Teach git-lfs to `GitCommitStore`
- `get_store_from_url` fails due to missing import `redis`
- Hardcoded version in `docs/conf.py`
- Use fsspec for store defintion HOT 1
- File-like interface of GoogleCloudStore fsspec implementation doesn't behave like previous releases HOT 2
- Add remaining tests to Github Actions
- Remove `BotoStore` and reliance on old `boto`
- Handle import of optional dependencies consistently
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from minimalkv.