Comments (5)
I was trying to come up with a solution how to automate this or make it less error-prone and have to write less manual code.
- Version wrapper enum:
enum StructVersions { V0(V0Struct), V1(V1Struct), Current(CurrentStruct), }
- The code you wrote could also be automatically generated by a proc macro:
The
#[pot_versioned(2)] struct CurrentStruct(..);
2
signifies the current version, so it will try to findV0Name
andV1Name
and generate the code you wrote above. Maybe it would generate a method calleddeserialize_versioned
for example. - Using a combination of both could make it easier to provide compatibility with other serializers. It could be used with
https://serde.rs/field-attrs.html#with
for example.
Just brain-dumping, hope you find any of it useful.
from pot.
so it will try to find V0Name and V1Name and generate the code you wrote above
My understanding of proc macros is that what you wrote is impossible. I was pretty sure that proc macros only can operate on the type they're being attached to and cannot find any other code outside of the TokenStream its associated with. If my assumption is correct, then CurrentStruct
's implementation can't actually find StructVersions, and even if it could, it couldn't look at the contents of StructVersions.
It could be used with https://serde.rs/field-attrs.html#with for example.
I don't think this is possible, because at the point that the with
function is being invoked, the only way to retrieve bytes is to call deserialize_bytes()
to retrieve the payload. In the current pushed code of dataversion, this isn't an issue because the callback receives a &[u8]
but in the new code, it receives a Read
generic -- allowing support for streaming dataversion'ed payloads. But, part of that is not knowing how long the input is.
If I proceed with allowing streaming deserialization of payloads, I don't think there's a way to use with.
I do think we can improve dataversion, but I'm also pretty sold on this being separate from Pot at this point. I don't have a good enough reason to leave it in Pot.
from pot.
If I proceed with allowing streaming deserialization of payloads, I don't think there's a way to use with.
I did -- and this is what I meant by deserializing receiving a Read generic. The complexity arises from needing to be able to peek-read the header, in case we decide it's not actually a dataversion
payload. Thus, we have to wrap the original Read
in a BufReader
to accomplish this, and then that resulting BufReader
is what must be used to deserialize the remaining data.
One bonus of how dataversion
is approaching this currently is that dataversion
can support migrating between Serde and non-Serde serialization frameworks. Start with rkyv
and want to move to serde
? No problem.
I think we can still improve dataversion
more without proc macros, but I also think that you're right -- ultimately to make this easy, we will need a proc macro.
from pot.
so it will try to find V0Name and V1Name and generate the code you wrote above
My understanding of proc macros is that what you wrote is impossible. I was pretty sure that proc macros only can operate on the type they're being attached to and cannot find any other code outside of the TokenStream its associated with. If my assumption is correct, then
CurrentStruct
's implementation can't actually find StructVersions, and even if it could, it couldn't look at the contents of StructVersions.
It can't find anything. But if you write #[dataversion(2)]
it can simply use V0Struct
and V1Struct
by simply using the name, if it can't find them it will just throw a compiler error. Basically was this does is use the fact that version names are standardized (they are not, we are requiring a specific version-naming-scheme for the proc-macro to work), to implement this.
We also don't need access to the contents, it can't implement the From
implementation, the user has to do that anyway. All it does is just build the match
statement. Basically, the user only has to write From
conversion for all versions and name them correctly, the rest is taken care of.
It could be used with https://serde.rs/field-attrs.html#with for example.
I don't think this is possible, because at the point that the
with
function is being invoked, the only way to retrieve bytes is to calldeserialize_bytes()
to retrieve the payload. In the current pushed code of dataversion, this isn't an issue because the callback receives a&[u8]
but in the new code, it receives aRead
generic -- allowing support for streaming dataversion'ed payloads. But, part of that is not knowing how long the input is.If I proceed with allowing streaming deserialization of payloads, I don't think there's a way to use with.
I had to read this a couple of times, lol, I'm very much out of the serialization game. The reason I'm proposing the enum
and #[serde(with = ...)]
is to make compatibility as easy as possible. That way users can use the same API as the always did, they just need to deserialize a different type, which then gets converted to the type they actually want.
Actually, thinking about it, if we generate this enum and use the usual de/serialize, I'm not sure what else we need to do.
The limitation you were describing, I have no clue really, sounded to me like we are trying to handle the version serialization ourselves, which I believe isn't necessary if the enum already contains that information.
I'm sry if I'm not understanding something here properly.
I do think we can improve dataversion, but I'm also pretty sold on this being separate from Pot at this point. I don't have a good enough reason to leave it in Pot.
That sounds great to me.
from pot.
After discussing on Discord we got on the same page. The process will be continued at khonsulabs/dataversion, although I may rename that crate depending on what all it tackles.
from pot.
Related Issues (6)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pot.