berlincode / ndb-orm Goto Github PK
View Code? Open in Web Editor NEWPython-NDB-ORM: ndb orm for use with google-cloud-datastore (python3 compatible)
License: Apache License 2.0
Python-NDB-ORM: ndb orm for use with google-cloud-datastore (python3 compatible)
License: Apache License 2.0
Looks like there is a conflict between the ndb_orm key module and the associated google.datastore.cloud key module.
Any timeline into integrating the ndb.Key functionality ?
I'm adding a KeyProperty to a model object and I'm getting an error about unicode type being not defined.
File "/usr/local/lib/python3.6/site-packages/ndb_orm/model.py", line 1961, in __init__ if isinstance(kind, unicode): NameError: name 'unicode' is not defined
I think most "legacy" GAE users will find this useful:
"""Base models and utilities used by the derived ones."""
import datetime
import logging
import ndb_orm as ndb
from google.cloud import datastore
from <project> import app, settings
# Datastore default client settings.
PROJECT = settings.PROJECT_ID
NAMESPACE = app.config["DATASTORE_NAMESPACE"]
NDB_KWARGS = {"project": PROJECT, "namespace": NAMESPACE}
# Module level singleton client used in all DB interactions. This is lazy inited when
# is used only, so we don't have any issues with Datastore agnostic tests/debugging,
# because creating a client will require valid credentials.
client = None
class BaseModel(ndb.Model):
"""Common model properties and functionality."""
# String used for properties with no available data.
NOT_SET = "N/A"
created_at = ndb.DateTimeProperty(auto_now_add=True)
def __init__(self, *args, **kwargs):
self._get_client() # Activates all NDB ORM required features.
kwargs.update(NDB_KWARGS)
super().__init__(*args, **kwargs)
@classmethod
def model_name(cls):
return cls.__name__.replace("Model", "")
@classmethod
def normalize(cls, value):
if value is None:
return cls.NOT_SET
return value
@staticmethod
def _get_client():
global client
if not client:
client = datastore.Client(**NDB_KWARGS)
ndb.enable_use_with_gcd(client=client, **NDB_KWARGS)
return client
@classmethod
def query(cls, **kwargs):
query = cls._get_client().query(kind=cls._get_kind(), **kwargs)
return query
@classmethod
def all(cls, query=None, keys_only=False, **kwargs):
query = query or cls.query()
query.order = ["-created_at"]
if keys_only:
query.keys_only()
return list(query.fetch(**kwargs))
@property
def myself(self):
"""Return the current DB version of the same object."""
return self.key.get()
@property
def exists(self):
"""Checks if the entity is saved into the Datastore."""
try:
return bool(self.myself) if self.key and self.key.id else False
except Exception:
return False
def put(self):
"""Saving the entity into the Datastore."""
self._get_client().put(self)
return self.key
@classmethod
def put_multi(cls, entities):
"""Multiple save in the DB without interfering with `cls.put` function."""
cls._get_client().put_multi(entities)
return [entity.key for entity in entities]
def remove(self):
"""Removes current entity and its dependencies (if any)."""
self.key.delete()
@classmethod
def remove_multi(cls, keys):
cls._get_client().delete_multi(keys)
@property
def urlsafe(self):
return self.key.to_legacy_urlsafe().decode(settings.ENCODING)
@classmethod
def get(cls, urlsafe_or_key):
if isinstance(urlsafe_or_key, (str, bytes)):
key = ndb.Key(cls, **NDB_KWARGS)
complete_key = key.from_legacy_urlsafe(urlsafe_or_key)
else:
complete_key = urlsafe_or_key
item = complete_key.get()
if not item:
raise Exception("item doesn't exist")
return item
Don't be sceptical, this is tested by these:
Just replace <project>
with your package name (under a Flask app for example).
I think I've spotted an issue and I'm not sure if this is the intended behaviour, line ~2935 in model.py:
@classmethod
def __get_arg(cls, kwds, kwd):
"""Internal helper method to parse keywords that may be property names."""
alt_kwd = '_' + kwd
if alt_kwd in kwds:
return kwds.pop(alt_kwd)
if kwd in kwds:
obj = getattr(cls, kwd, None)
if not isinstance(obj, Property) or isinstance(obj, ModelKey):
return kwds.pop(kwd)
return None
more exactly: if not isinstance(obj, Property) or isinstance(obj, ModelKey):
Shouldn't this actually be: if not (isinstance(obj, Property) or isinstance(obj, ModelKey)):
in order to pop out the potential keyword argument from the kwds
dict? I think the logic of this is actually the following:
If the requested keyword argument name isn't by any chance an already existing property of the model (as
Property
or the special oneModelKey
), only then retrieve & return the value (and pop the key out ofkwds
). Otherwise, we need to keep it in, because the value belongs to the property and it's not a parameter of the model initializer.
Or, for example in the key
param case, I may be totally wrong, because ModelKey
is subclass of Property
and requesting it from the kwds should actually return it, but will fail in the first check, that's why there's the right side of the or
which checks for this special property in order to return it (and pop it out from the properties), so the key can be post-processed and set inside the __init__
method, rather than being passed to the properties setting method furthermore.
I'm trying to directly instantiate a ndb.Key() to use with my Entity object.
However looking at the source code, this instantiates a KeyBase class which is not yet implemented.
I'm not sure if you have a different version of this class.
raise NotImplementedError('KeyBase class is not set up (yet)') NotImplementedError: KeyBase class is not set up (yet)
Instantiated from KeyClass
line 49.
I need to setup multi tenancy using the namespace concept of the Cloud DataStore.
Inspecting the code, I'm not sure this is fully implemented within the current version of the library.
Can you help me understand what is missing to support that ? Thanks
Hi @berlincode ,
We are switching from GAE standard to GAE flexible. I'm looking for a good NDB replacement and we need performance.
For the C10K problem Python3 has asyncio that provides high performance by event loop and asynchronous I/O.
Why doesn't your ndb-orm have _async variants?
How do we make ndb-orm to support fully async operations?
I found this one. Is it possible to combine ndb-orm with it?
The ndb-orm/query.py
file is missing. Running any query with Model.query()
results in error:
File "/home/.../venv/lib/python3.6/site-packages/ndb_orm/model.py", line 3420, in _query
from .query import Query # Import late to avoid circular imports.
ModuleNotFoundError: No module named 'ndb_orm.query'
README.md says:
This is not a drop-in replacement for the whole ndb package (no ndb.context, ndb.tasklets, ndb.query or ndb.Key), but allows you to use ndb.model classes.
So the file might be intentionally omitted. This code throws an exception when properly used though.
How to reproduce:
Person
Person.query()
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.