Comments (8)
That's very interesting, I have no clue what's happening here. I'll try and investigate it soon. Is this crucial for your use-case, or can it wait a bit? I have a few other projects I'm working on plus I'm moving on Sunday, so I'm a bit busy currently.
from jsonpickle.
It can definitely wait. I really appreciate your asking! Good luck with the move -- that's always a big project!!
from jsonpickle.
It seems like it was an intentional choice by the maintainers before me to have circular objects become None when pickled with unpickleable=False
:
if self.unpicklable:
...
else:
max_reached = self._max_reached()
in_cycle = _in_cycle(obj, self._objs, max_reached, False)
if in_cycle:
# A circular becomes None.
return None
Found here
While I'd support changing the behavior to be different by default, I'm afraid this would have to wait for jsonpickle 4, as changing this would be breaking backwards compatibility. Does your use case for this require different behavior when unpickleable=False
, and if so, would adding an option to the encoder to try and better represent circular objects be useful? The latter option could be backwards-compatible, and could come out with jsonpickle 3.1.0, which I was planning on releasing within a month or two.
from jsonpickle.
I don't think datetime.timedelta
is circular. I stepped through this code into the call to jsonpickle.encode()
from datetime import timedelta
import jsonpickle
def main():
td = timedelta(seconds=1)
print(f'{td=}')
td_json_str = jsonpickle.encode(td, unpicklable=False)
print(f'{td_json_str=}')
if __name__ == '__main__':
main()
and the return None
is here (line 664):
File: pickler.py
658: # catchall return for data created above without a return
659: # (e.g. __getnewargs__ is not supposed to be the end of the story)
660: if data:
661: return data
662:
663: self._pickle_warning(obj)
664: return None
I'm wondering if perhaps the issue (no data
at line 660) is that timedelta
has __reduce__
and __reduce_ex__
but does not have __getstate__
?
from jsonpickle.
I don't understand why only the unpicklable=True
case calls __reduce__
, whereas unpicklable=False
only calls __getstate__
, but apparently that is how _flatten_obj_instance
was written. Is there a design document or some history to the project which records that decision?
Once there's a release with the fix for #443, I intend to use unpicklable=True
(in other words, I only turned to unpicklable=False
to try to find a temporary workaround.) But I would have thought that the code to spelunk an object to record its state would perhaps share a common core for both states of unpicklable
.
from jsonpickle.
Unfortunately there is no official design document/history to know why these choices were made. I only started working on jsonpickle in late 2020, and jsonpickle started in 2009, so I don't really know the reasoning behind much of the code-base. Maybe @davvid can provide some insights?
My best guess is that since __reduce__
is generally used by the python cPickle module to serialize objects it otherwise couldn't, it has a usage tied to (de)serializing objects, where __getstate__
is used more for debugging purposes. I personally mainly use __getstate__
for debugging, and have never used __reduce__
for anything other than serialization, so maybe the original authors thought of it this way also.
from jsonpickle.
I guess my main point is that there are builtin types, like timedelta
that simply do not have a __getstate__
(before Python 3.11) but do have a __reduce__
. Even with Python 3.11, it isn't clear to me what the purpose of the default object
's __getstate__
is. For example
Python 3.11.3 (tags/v3.11.3:f3909b8, Apr 4 2023, 23:49:59) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import timedelta
>>> td = timedelta(seconds=3)
>>> td
datetime.timedelta(seconds=3)
>>> td.__getstate__()
>>> td.__reduce__()
(<class 'datetime.timedelta'>, (0, 3, 0))
It seems to me that jsonpickle
should prefer to use __reduce__
if available, but I admit I'm not clear on what the role of these two functions are meant to be. I'm assuming there's some PEP or other document that explains this?
from jsonpickle.
Maybe related #447
from jsonpickle.
Related Issues (20)
- .asc missing on pypi for 3.0.2 HOT 2
- Complete compliance steps in Tidelift HOT 4
- RFE: is it possible to start making github releases?🤔 HOT 4
- Project's URL is not up-to-date HOT 3
- wrong encoding for decorator objects HOT 1
- Converts objects of a subclass to a superclass HOT 1
- Register handlers for dicts HOT 6
- Error deserializing django SafeString HOT 6
- Add a "Usage" section to the README HOT 8
- 10 tests fail HOT 6
- Possible problem with encoding in a Aysnchronous progam when calling a task HOT 4
- Fuzz Testing via OSS-Fuzz HOT 8
- Fails trying to unpickle a Pandas Timedelta object HOT 4
- Clarification on dataclasses, functools.partial, py/state HOT 5
- Nested dictionaries in `py/state` incorrectly restored when tracking object references
- Dictionaries that use tuples as keys get the key restored as string not a tuple HOT 4
- AttributeError: 'Int64' object attribute '_type_marker' is read-only HOT 2
- Python 3.13 regressions HOT 2
- mistaken
- RFC: String quoting in jsonpickle's code HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jsonpickle.