Serialize issue of ObjectId

[thedodd]

Serialize issue of ObjectId

[ndelvalle]

@surfingtomchen the approach I am using is this one:

use serde::Serializer;
use wither::bson::oid::ObjectId;

pub fn serialize_object_id<S>(object_id: &Option<ObjectId>, serializer: S) -> Result<S::Ok, S::Error>
where
    S: Serializer,
{
    match object_id {
      Some(ref object_id) => serializer.serialize_some(object_id.to_string().as_str()),
      None => serializer.serialize_none()
    }
}

#[derive(Debug, Model, Serialize, Deserialize)]
pub struct Foo {
    #[serde(
        rename = "_id",
        skip_serializing_if = "Option::is_none",
        serialize_with = "serialize_object_id"
    )]
    pub id: Option<ObjectId>,
}

I am not sure if this is the best approach but you can try it out.

[saghm]

@surfingtomchen I think you might be misunderstanding the output when you're printing a BSON document. Try running the following code in a clean Rust project:

[dependencies]
mongodb = "1.1.1"
serde = { version = "1.0.116", features = ["derive"] }
tokio = { version = "0.2.22", features = ["macros", "rt-threaded"] }

use mongodb::{bson::oid::ObjectId, Client};
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize, Clone)]
struct User {
    /// The ID of the model.
    #[serde(rename = "_id", skip_serializing_if = "Option::is_none")]
    id: Option<ObjectId>,

    nick: String,
}

#[tokio::main]
async fn main() {
    let client = Client::with_options(Default::default()).unwrap();
    let coll = client.database("foo").collection("bar");

    let user = User {
        id: Some(ObjectId::new()),
        nick: "saghm".into(),
    };
    coll.insert_one(mongodb::bson::to_document(&user).unwrap(), None)
        .await
        .unwrap();

    let user = coll.find_one(None, None).await.unwrap().unwrap();

    println!("{:#?}", user);
}

The output of this short program will show that the ObjectId is correctly getting round-tripped as a bson ObjectId value:

Document({"_id": ObjectId(ObjectId(5f60410c00efcb5f004b12a3)), "nick": String("saghm")})

The confusion you're having stems from the fact that our Display implementation for BSON uses JSON formatting for the values. However, JSON doesn't have many of the types that BSON has, including ObjectId. If we used "_id": "5f5ae5b400f0368a00715eca" like you suggest in your original issue description, it would be indistinguishable from if the _id field was actually a BSON string rather than an ObjectId. To alleviate this, we use a format called Extended JSON, which defines how to represent non-JSON values using JSON objects with $-prefixed keys. The extended JSON definition of ObjectId is to use an object with a single $oid field with a hex-encoded value representing the bytes it contains. To be clear, the data is being sent to the database as a raw ObjectId, not as a document with the $oid field; you can verify this by connecting to the database with the shell and inspecting the data manually.

[surfingtomchen]

@thedodd @saghm Thank you for your clarification. I totally understand and agree with you why the ObjectId should be displayed as Extended JSON with $oid inside it to have Mongo read it properly.

I guess maybe current issue should not be described as bug but a feature request. Because when I use the mongoose for Node.js, if I defined id as ObjectId, mongoose will automatically displayed as string once output it to the http response. That's why @thedodd suggest me to use another structure to store the value as String. It will solve the problem but have a little bit inconvience.

I believe Wither plays the same role in Rust as Mongoose to Node.js. Please consider the feature request, thanks.

[surfingtomchen]

@surfingtomchen good, glad we were able to find an immediate solution. For a long-term change as you mentioned, I think it would be best to open an issue for this in the BSON crate, as that is where the type lives. I leave that up to you. It may need some design, perhaps a special function ... not sure.

Let me know if there is anything else to discuss. Otherwise, mind closing the issue?

Yes, the issue will be closed

[thedodd]

@surfingtomchen are you getting an actual error somewhere in your code? {"$oid": "..."} is how BSON encodes an ObjectId, so I don't necessarily see a problem here.

[surfingtomchen]

    "data": {
        "_id": {
            "$oid": "5f5ae5b400f0368a00715eca"
        },
        "nick": "Tm999y"
    }

how to convert to

    "data": {
        "_id":  "5f5ae5b400f0368a00715eca",
        "nick": "Tm999y"
    }

I tried flatten of serde, but it shows

    "data": {
        "$oid":  "5f5ae5b400f0368a00715eca",
        "nick": "Tm999y"
    }

and can not rename after flattern

[thedodd]

@surfingtomchen I understand. What I am trying to communicate however is that the representation you are seeing is correct. When you encode a BSON ObjectId as extended JSON, as described here, that is what you get.

What you are seeing is an issue with this BSON library (https://github.com/mongodb/bson-rust). There is probably lots more discussion on this very topic over there.

HOWEVER, an immediate solution would be to create a struct which holds your ObjectId field as a String, and then when you write that struct as JSON, it will render the way that you want it to.

That should resolve your issue. If you think changes are needed to the BSON lib, please open an issue there. Cheers!

[thedodd]

@surfingtomchen good, glad we were able to find an immediate solution. For a long-term change as you mentioned, I think it would be best to open an issue for this in the BSON crate, as that is where the type lives. I leave that up to you. It may need some design, perhaps a special function ... not sure.

Let me know if there is anything else to discuss. Otherwise, mind closing the issue?