Comments (30)
I have a rather pressing need for this on a work project that's experimenting heavily with diesel, and I can almost certainly talk my boss into letting me work on this on company time extremely soon. (We really like diesel so far! But we want to use it for more things.)
Probably the nicest and richest JSON representation in Rust right now is serde_json::Value
. It's supported by lots of other libraries, and you can serialize and deserialize it in either direction:
raw JSON String
or Vec<u8>
β serde_json::Value
β Rust data structure implementing Serialize
or Deserialize
It's super handy. The rust-postgres
crate has some code showing how to set this up.
Of course, this could be put under a serde_json
feature, like the Uuid
and chrono
types are.
Should I just go ahead and bang out a PR showing what I mean?
from diesel.
What's the work involved in supporting any serde_json::Serializable
vs. just serde_json::Value
? Happy to help out if I can.
The use case is an API client that returns a struct. The struct is serializable, but isn't a serde_json::Value
, so to store the raw value I need to re-serialize, then re-de-serialize into a generic serde_json::Value
.
from diesel.
I would like to use diesel for a project at my job, and we use jsonb
extensively. I would like to take on the project of implementing this, but I would need some help getting started.
from diesel.
That part was quoting you from above:
So let's talk borderline absurd features.
:)
from diesel.
PR #561 was merged, but this is still open. (For the record, Jsonb support can be enabled with the serde_json
feature.)
It looks like the basic support is here, but without the operators, right?
from diesel.
Any movement on this?
It's really crucial for me.
Most important would be just supporting fields with a "Value" type from serde_json.
from diesel.
@sgrif have you seen the rust-postgres-derive crate? "Syntax extensions to automatically derive FromSql and ToSql implementations for Postgres enum, domain, and composite types." https://github.com/sfackler/rust-postgres-derive
Something like it might make sense for Diesel.
from diesel.
I'm fine with having it there as long as this can be done by just adding a bunch of impl to diesel itself (behind the serde_json
feature flag) and as long as someone contributes that code + some tests.
from diesel.
I need this too. This and Tsvector get hung up. How difficult is this to implement? It looks like I may have to drop down to rust-postgres...
from diesel.
And just so it's clear, if we do this right, we should be able to have anything which implements serde::ser::Serialize
implement AsExpression<Json>
via ToSql<Json>
from diesel.
So, I was just trying to use Jsonb in a table and I had kind of assumed--until I ran into this error--that diesel would give the json to me as a string and I could deserialize it myself. No dice. Instead, I get a compilation error. Rather than wait for the absurd feature (querying jsonb with diesel, I guess?), what about just letting diesel give me the data instead of exploding? :)
from diesel.
Yes, you can fairly easily add support for any types that you want within
your application. I'm on my phone but will give examples later
Rather than wait for the absurd feature
Unsure what that is supposed to mean
On Tue, May 10, 2016, 5:02 PM archer884 [email protected] wrote:
So, I was just trying to use Jsonb in a table and I had kind of
assumed--until I ran into this error--that diesel would give the json to me
as a string and I could deserialize it myself. No dice. Instead, I get a
compilation error. Rather than wait for the absurd feature (querying jsonb
with diesel, I guess?), what about just letting diesel give me the data
instead of exploding? :)β
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#44 (comment)
from diesel.
Oh. I didn't realize this was a comment on an existing issue. XD
from diesel.
So basically what you'd need to do is:
struct JsonB;
impl HasSqlType<JsonB> for Pg {
// ...
}
impl FromSql<JsonB, Pg> for String {
// ...
}
expression_impls!(JsonB -> String);
Unsure what the binary rep of jsonb is so you might need to do some digging there
from diesel.
@archer884 Did you end up implementing this as Sean described? I have a table with a jsonb column and was hoping someone had already done the heavy lifting of creating a Diesel type for it that I could copy/paste. :}
from diesel.
Wow, I hadn't realized how many people want to have this! :) @emk, it would be amazing if you got company time to work on this! Maybe you, @archer884 and @norcalli can all work together on this!
Should I just go ahead and bang out a PR showing what I mean?
Maybe. Or, you could try and do this as a plugin crate. (If you think adding this to diesel is easier and more efficient, you can of course still open a PR!)
Diesel has a bunch of facilities to make implementing custom types easier, and as @sgrif mentions in the issue description, it may be possible to add this in a separate crate. This may be a worthwhile effort as it could also show that it is possible to write plugins for diesel and how to do it. Currently, there is just diesel_full_text_search that I know of (and it doesn't even have a Readme⦠I just field diesel-rs/diesel_full_text_search#1).
(Initially I was worried how quickly you'll run into the limits of the current orphan rules, but I think you'll be fine if you define a custom Jsonb
type and impl all traits on this, or as extension traits.)
from diesel.
@killercup Thank you for the encouragement!
I've opened a "work in progress" PR at #561, just so we can visualize what I'm thinking about. I agree that it would be an interesting possibility to do this as a separate plugin, but I'd prefer to try that after we see it working inside diesel
itself!
Also, for the first version, I'm just targeting serialization and deserialization, and not the specialized JSON query operators. If we can begin by getting JSON into and out of the database, that will already help with a lot of use cases.
Anyway, anybody who wants to help out is encouraged to check out #561, and submit comments, bug reports, and further PRs. :-) I'll continue the implementation discussion there.
from diesel.
I now have a preliminary implementation of the json
and jsonb
data types and conversion to and from serde_json::Value
at #561! I haven't attempted to implement any of the JSON-specific query operators.
See #561 for instructions on how to try this out, and please let me know if it works!
from diesel.
Hey folks -- today's my first day back from holiday vacation. I'm getting caught up on issues now. I've gone ahead and added this feature to the 0.10 timeline, as there's clearly a lot of demand for it.
https://github.com/diesel-rs/diesel_full_text_search was mostly just a proof of concept to demonstrate some of the basics of how to add support for additional extensions outside of Diesel. It "conveniently" tackled some types which didn't require ToSql
/FromSql
implementations. I'd like to see a plugin crate done to ensure we have appropriate APIs in place, but I'm fine with adding this to Diesel proper for now and exploring that space at a later date.
I'll leave implementation specific comments on #561.
from diesel.
@dbrgn exactly. We should probably make this a meta issue with a check list of stuff we want to have.
from diesel.
Was wondering if any progress has been made implementing json operators e.g. ->
and such yet? If not, I'd love to try to implement this as I'm very interested in seeing this functionality in Diesel
from diesel.
@jonnybrooks As far as I know there is no process in those operators made. Feel free to try to implement it. If you hit any problem just ask in out gitter room.
from diesel.
Hello,
First thanks for diesel, it's amazing !
I can't manage to use the @>
operator on a Jsonb column with the method contains
. I thought is was implemented based on @sgrif original comment but I can't found any trace of it in the docs (even when generated with feature serde_json
).
Here's my diesel dependency in my Cargo.toml:
diesel = { version = "1.4.2", features = ["chrono", "postgres", "r2d2", "serde_json"] }
An extract of the code:
use crate::schema;
use diesel::{dsl, prelude::*};
// Some code and in a function I end up having this where `integration_data` is of type
// `serde_json::Value`:
let filter = schema::item_characteristics::item_id
.eq(schema::items::id)
.and(schema::item_characteristics::integration_data.contains(integration_data));
The associated schema.rs:
table! {
items (id) {
id -> Int4,
retailer_id -> Int4,
quantity -> Int4,
try_fill -> Bool,
distributed_price_et -> Nullable<Float8>,
public_price_et -> Float8,
non_promo_public_price_et -> Float8,
sr_block_distribution -> Bool,
sr_block_fill -> Bool,
refreshed_at -> Timestamptz,
updated_at -> Timestamptz,
created_at -> Timestamptz,
locked_at -> Nullable<Timestamptz>,
}
}
table! {
item_characteristics (item_id) {
item_id -> Int4,
owned_by_retailer -> Bool,
seen_in_retailer_inventory -> Bool,
collection_started_at -> Nullable<Timestamptz>,
integration_data -> Jsonb,
updated_at -> Timestamptz,
}
}
And finally the compilation error:
error[E0599]: no method named `contains` found for type `schema::item_characteristics::columns::integration_data` in the current scope
--> /Users/elrendio/Workspace/Stockly/Main/Stocks/src/items/lock.rs:86:58
|
86 | .and(schema::item_characteristics::integration_data.contains(integration_data)),
| ^^^^^^^^
|
::: /Users/elrendio/Workspace/Stockly/Main/Stocks/src/schema.rs:87:1
|
87 | / table! {
88 | | item_characteristics (item_id) {
89 | | item_id -> Int4,
90 | | owned_by_retailer -> Bool,
... |
95 | | }
96 | | }
| |_- method `contains` not found for this
|
= note: the method `contains` exists but the following trait bounds were not satisfied:
`&mut schema::item_characteristics::columns::integration_data : diesel::PgArrayExpressionMethods`
`&schema::item_characteristics::columns::integration_data : diesel::PgArrayExpressionMethods`
`schema::item_characteristics::columns::integration_data : diesel::PgArrayExpressionMethods`
= help: items from traits can only be used if the trait is implemented and in scope
= note: the following traits define an item `contains`, perhaps you need to implement one of them:
candidate #1: `std::ops::RangeBounds`
candidate #2: `diesel::PgArrayExpressionMethods`
candidate #3: `percent_encoding::EncodeSet`
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
If it's not implemented my company would gladly invest some time to contribute provided some basic guidelines on how to do it.
Thanks a lot !
from diesel.
@Elrendio The list in Seans original comment is more or less a wish list. As far as I'm aware none of those operators are implemented yet.
That said: Diesel provides the diesel_infix_operator!
which allows to define that operator easily on yourself.
Also if someone finds some time to implement them in a proper way (like PgArrayExpressionMethods
here) feel free to submit a PR.
from diesel.
Thank you @weiznich, we'll look at the PgArrayExpressionMethods
and try to open a PR in the next few weeks :)
from diesel.
@weiznich I am sorry if I sound stupid, but would you just add a simple example of how this macro diesel_infix_operator!()
works to filter json columns by fields in postgres?
from diesel.
@gh67uyyghj The api documentation already has an example.
from diesel.
It seems that part of what is being proposed here (#44 (comment)) contradicts with #1950 (comment).
from diesel.
@cameron-martin Feel free to provide a PR adding the corresponding impl's for allowing that without relying on any macro additionally to the normal diesel/serde derives to be called there.
from diesel.
Does this mean your position of not wanting this in diesel core (#1950 (comment)) has changed?
from diesel.
Related Issues (20)
- SSL_MODE option not behaving correctly with Mysql 5.7 HOT 3
- Derive Queryable deserialize_as attribute doesn't support tuples HOT 4
- msvc+rust1.75 compile error HOT 4
- `Using unsupported buffer type: 253 (parameter: 1)` when trying to use the CLI HOT 14
- double free or corruption issue when establishing connection HOT 18
- Unable to group by date HOT 1
- Regression: LIKE queries on nullable fields in MySQL HOT 5
- Windows CI is failing to install postgres HOT 2
- Can not use .eq(Decimal_field + xxx) HOT 4
- Closing a sqlite connection does not clear the -wal and -shm file HOT 4
- Execute single-statement migrations without batch_execute HOT 6
- Linking error for diesel_cli in Alpine despite libraries being installed HOT 5
- Option field check in diesel_derives in incomplete
- oid caching leads to broken queries after schema changes HOT 1
- The diesel 2.1.5 version cannot close the tls connection HOT 7
- Diesel-Schema not supporting Postgres-SCHEMA HOT 1
- `skip_insertion` results in a compiler error HOT 3
- table with more than one primary_key can not use on_conflict().do_update() HOT 1
- Make empty changeset updates using `.optional()` , either ignore or bypass errors `QueryBuilderError` s.
- Rust-analyzer fails to autocomplete Match with diesel errors (Enum) HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diesel.