Do we really need `ort::session::InMemorySession`?

[pykeio]

Do we really need `ort::session::InMemorySession`?

[SunDoge]

I see. But by default

If a session is created using an input byte array containing the ORT format model data, by default we will copy the model bytes at the time of session creation to ensure the model bytes buffer is valid.

The api with_model_from_memory should be consistent with with_model_from_file. And ort::session::InMemorySession should be created with another api, e.g. with_model_from_memory_directly.

[SunDoge]

And the C API does not mention that we have to keep model_bytes alive while using Session. Session should parse the model_bytes and keep the parsed params and graph in memory so it can outlive the model_bytes.

[decahedron1]

Session should parse the model_bytes and keep the parsed params and graph in memory so it can outlive the model_bytes.

Not always, For .ort models, we enable session.use_ort_model_bytes_directly:

You may also enable the option to use the model bytes directly by setting the SessionOptions config entry session.use_ort_model_bytes_directly to 1. This may reduce the peak memory usage of ONNX Runtime Mobile, but you will need to guarantee that the model bytes are valid throughout the lifespan of the ORT session.

https://onnxruntime.ai/docs/performance/model-optimizations/ort-format-models.html