kubo / rust-oracle Goto Github PK
View Code? Open in Web Editor NEWOracle driver for Rust
Oracle driver for Rust
Should this work with timesten? I'm not having much luck - although I get the feeling it should
Hi,
as the title says.
During profiling of retrieving large sets of data from Oracle databases, I noticed that <Row as RowValue>::get()
prominently stands out if there are lots of small rows.
For each row, the value handles are duplicated - is there any specific reason for this? If there is, than the following is probably wrong and I'd have to accept the sizeable performance hit.
I created a small patch which shares the values across row, which worked fine so far.
Thanks,
Christoph
How would I translate the following connection string?
jdbc:oracle:thin:@//127.0.0.1:1522/some.oracle.service-name.org
background: i wanted to implement an r2d2-oracle
adapter between the r2d2 crate and your crate.
this was tested using
the struct oracle::Connection
does not implement the Sync
and Send
traits. these traits are usually automatically given to all struct
s but this is not happening here since Connection
has raw pointers (handle
).
to reproduce:
use std::thread;
use oracle::Connection;
fn main() {
let connection = Connection::connect("username", "password", "connect_string", &[]).unwrap();
thread::spawn(move || {
connection.query("SELECT 1 FROM dual", &[]).unwrap();
});
}
this gives the following compile errors:
error[E0277]: `*mut oracle::binding::dpiConn` cannot be sent between threads safely
--> src\main.rs:7:5
|
7 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `*mut oracle::binding::dpiConn` cannot be sent between threads safely
|
= help: within `[closure@src\main.rs:7:19: 9:6 connection:oracle::connection::Connection]`, the trait `std::marker::Send` is not implemented for `*mut oracle::binding::dpiConn`
= note: required because it appears within the type `oracle::connection::Connection`
= note: required because it appears within the type `[closure@src\main.rs:7:19: 9:6 connection:oracle::connection::Connection]`
= note: required by `std::thread::spawn`
error[E0277]: `*mut oracle::binding::dpiContext` cannot be shared between threads safely
--> src\main.rs:7:5
|
7 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `*mut oracle::binding::dpiContext` cannot be shared between threads safely
|
= help: within `oracle::Context`, the trait `std::marker::Sync` is not implemented for `*mut oracle::binding::dpiContext`
= note: required because it appears within the type `oracle::Context`
= note: required because of the requirements on the impl of `std::marker::Send` for `&'static oracle::Context`
= note: required because it appears within the type `oracle::connection::Connection`
= note: required because it appears within the type `[closure@src\main.rs:7:19: 9:6 connection:oracle::connection::Connection]`
= note: required by `std::thread::spawn`
error[E0277]: `*const i8` cannot be shared between threads safely
--> src\main.rs:7:5
|
7 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `*const i8` cannot be shared between threads safely
|
= help: within `oracle::Context`, the trait `std::marker::Sync` is not implemented for `*const i8`
= note: required because it appears within the type `oracle::binding::dpiCommonCreateParams`
= note: required because it appears within the type `oracle::Context`
= note: required because of the requirements on the impl of `std::marker::Send` for `&'static oracle::Context`
= note: required because it appears within the type `oracle::connection::Connection`
= note: required because it appears within the type `[closure@src\main.rs:7:19: 9:6 connection:oracle::connection::Connection]`
= note: required by `std::thread::spawn`
error[E0277]: `*mut oracle::binding::dpiAppContext` cannot be shared between threads safely
--> src\main.rs:7:5
|
7 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `*mut oracle::binding::dpiAppContext` cannot be shared between threads safely
|
= help: within `oracle::Context`, the trait `std::marker::Sync` is not implemented for `*mut oracle::binding::dpiAppContext`
= note: required because it appears within the type `oracle::binding::dpiConnCreateParams`
= note: required because it appears within the type `oracle::Context`
= note: required because of the requirements on the impl of `std::marker::Send` for `&'static oracle::Context`
= note: required because it appears within the type `oracle::connection::Connection`
= note: required because it appears within the type `[closure@src\main.rs:7:19: 9:6 connection:oracle::connection::Connection]`
= note: required by `std::thread::spawn`
error[E0277]: `*mut std::ffi::c_void` cannot be shared between threads safely
--> src\main.rs:7:5
|
7 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `*mut std::ffi::c_void` cannot be shared between threads safely
|
= help: within `oracle::Context`, the trait `std::marker::Sync` is not implemented for `*mut std::ffi::c_void`
= note: required because it appears within the type `oracle::binding::dpiConnCreateParams`
= note: required because it appears within the type `oracle::Context`
= note: required because of the requirements on the impl of `std::marker::Send` for `&'static oracle::Context`
= note: required because it appears within the type `oracle::connection::Connection`
= note: required because it appears within the type `[closure@src\main.rs:7:19: 9:6 connection:oracle::connection::Connection]`
= note: required by `std::thread::spawn`
error[E0277]: `*mut oracle::binding::dpiPool` cannot be shared between threads safely
--> src\main.rs:7:5
|
7 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `*mut oracle::binding::dpiPool` cannot be shared between threads safely
|
= help: within `oracle::Context`, the trait `std::marker::Sync` is not implemented for `*mut oracle::binding::dpiPool`
= note: required because it appears within the type `oracle::binding::dpiConnCreateParams`
= note: required because it appears within the type `oracle::Context`
= note: required because of the requirements on the impl of `std::marker::Send` for `&'static oracle::Context`
= note: required because it appears within the type `oracle::connection::Connection`
= note: required because it appears within the type `[closure@src\main.rs:7:19: 9:6 connection:oracle::connection::Connection]`
= note: required by `std::thread::spawn`
error[E0277]: `*mut oracle::binding::dpiShardingKeyColumn` cannot be shared between threads safely
--> src\main.rs:7:5
|
7 | thread::spawn(move || {
| ^^^^^^^^^^^^^ `*mut oracle::binding::dpiShardingKeyColumn` cannot be shared between threads safely
|
= help: within `oracle::Context`, the trait `std::marker::Sync` is not implemented for `*mut oracle::binding::dpiShardingKeyColumn`
= note: required because it appears within the type `oracle::binding::dpiConnCreateParams`
= note: required because it appears within the type `oracle::Context`
= note: required because of the requirements on the impl of `std::marker::Send` for `&'static oracle::Context`
= note: required because it appears within the type `oracle::connection::Connection`
= note: required because it appears within the type `[closure@src\main.rs:7:19: 9:6 connection:oracle::connection::Connection]`
= note: required by `std::thread::spawn`
error: aborting due to 7 previous errors
For more information about this error, try `rustc --explain E0277`.
as a workaround a wrapper struct can be implemented which explicitly implements Send
and Sync
:
use std::ops::Deref;
use std::thread;
use oracle::Connection;
struct OracleConnection(oracle::Connection);
impl Deref for OracleConnection {
type Target = oracle::Connection;
fn deref(&self) -> &oracle::Connection {
&self.0
}
}
unsafe impl Send for OracleConnection {}
unsafe impl Sync for OracleConnection {}
fn main() {
let connection = Connection::connect("username", "password", "connect_string", &[]).unwrap();
let connection = OracleConnection(connection);
thread::spawn(move || {
connection.query("SELECT 1 FROM dual", &[]).unwrap();
});
}
however i don't know if this is really safe since i don't know the internals of your library.
see also:
do you have a roadmap to releasing a v1.0.0? it would be cool if you could create the tickets for what you think is missing for v1.0.0 and assign them to a v1.0.0 milestone.
rationale: theoretically, one should only build on released software, so using 0.x pre-releases for production software isn't looked on too well. in the rust eco-system there are sadly a lot of crates which stay on 0.x for years but are heavily used. having a roadmap helps in understanding why a crate is not yet released as 1.x (or higher) and allows analysing the trade-offs (is it acceptable to use the crate in production even though these features are missing?).
furthermore, this would help with contributors: they know where work still needs to be done and they can offer their help focused on these topics.
Hello, and thanks a lot for your work.
I'd like to replace the following JDBC code:
val updateFlags = conn.prepareStatement("UPDATE table SET field='something' WHERE id = ?")
ids.toSet().forEach { id ->
updateFlags.setLong(1, id)
updateFlags.addBatch()
}
updateFlags.executeBatch()
updateFlags.close()
Which runs decently fast (a few hundred ms for 10k entries)
I naively tried to run:
let now = Instant::now();
let mut stmt = conn.prepare("UPDATE table SET field='something' WHERE id = :1", &[StmtParam::FetchArraySize(10_000)])?;
for id in &ids {
stmt.execute(&[id])?;
}
println!("Time for {} updates: {}", ids.len(), now.elapsed().as_millis());
conn.commit()
But unfortunately it takes way too much time as the number of ids
increase. (More than 10s for 10k entries).
I see in README there's "Batch support" mentioned.
I guess this covers such feature, am I right?
If so, well, thank you!
And could you please notify me by closing this issue whenever you have time to implement it?
This way I would know I can go back to dealing with Oracle in Rust!
Thanks a lot.
I've tried the following code:
use oracle::*; // oracle = "0.5.6"
const CREATE_TEST_TABLE: &str = "CREATE TABLE test_table (\
text clob
)";
fn repro(conn: Connection) {
let _ = conn.execute("DROP TABLE test_table", &[]);
conn.execute(CREATE_TEST_TABLE, &[]).unwrap();
let mut stmt = conn
.statement("INSERT INTO test_table(text) VALUES (:in1) RETURNING text INTO :out1")
.build()
.unwrap();
let long_text = std::iter::repeat('a').take(4000).collect::<String>();
stmt.execute_named(&[("in1", &long_text), ("out1", &None::<String>)])
.unwrap();
let s: &String = &stmt.returned_values("out1").unwrap()[0];
assert_eq!(s, &long_text);
}
I expect that this code passes without an error for a valid connection. Instead of that I get the following error message:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: OciError(DbError { code: 22835, offset: 53, message: "ORA-22835: Buffer too small for CLOB to CHAR or BLOB to RAW conversion (actual: 4000, maximum: 2000)", fn_name: "dpiStmt_execute", action: "execute" })', src/main.rs:16:10
(where line main.rs:16
refers to the unwrap after the execute_named
)
I have a program which iterates through millions of rows, some tables having ~50 columns and doing this through the existing functions seems to be quite slow when there are a lot of columns.
In a similar program written in Java I switched to using REF CURSORs through the JDBC driver and this drastically improved performance. I'd like to be able to use the same in Rust.
mismatched types error when building for ARM aarch64. I find a similar issue from remacs repo
error[E0308]: mismatched types
--> src/batch.rs:206:56
|
206 | dpiStmt_getBindNames(handle, &mut num, names.as_mut_ptr(), lengths.as_mut_ptr()),
| ^^^^^^^^^^^^^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*mut *const u8`
found raw pointer `*mut *const i8`
error[E0308]: mismatched types
--> src/batch.rs:213:45
|
213 | bind_names.push(to_rust_str(names[i], lengths[i]));
| ^^^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*const u8`
found raw pointer `*const i8`
error[E0308]: mismatched types
--> src/sql_type/object.rs:199:45
|
199 | dpiData_setBytes(&mut data, buf.as_mut_ptr(), buf.len() as u32);
| ^^^^^^^^^^^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*mut u8`
found raw pointer `*mut i8`
error[E0308]: mismatched types
--> src/sql_type/object.rs:402:45
|
402 | dpiData_setBytes(&mut data, buf.as_mut_ptr(), buf.len() as u32);
| ^^^^^^^^^^^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*mut u8`
found raw pointer `*mut i8`
error[E0308]: mismatched types
--> src/sql_value.rs:532:63
|
532 | dpiLob_readBytes(lob, offset, READ_CHAR_SIZE, bufptr, &mut read_len)
| ^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*mut u8`
found raw pointer `*mut i8`
error[E0308]: mismatched types
--> src/sql_value.rs:555:17
|
555 | result.as_mut_ptr() as *mut i8,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*mut u8`
found raw pointer `*mut i8`
error[E0308]: mismatched types
--> src/sql_value.rs:582:58
|
582 | dpiLob_readBytes(lob, offset, READ_SIZE, bufptr, &mut read_len)
| ^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*mut u8`
found raw pointer `*mut i8`
error[E0308]: mismatched types
--> src/sql_value.rs:671:21
|
671 | self.keep_bytes.as_mut_ptr() as *mut i8,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*mut u8`
found raw pointer `*mut i8`
error[E0308]: mismatched types
--> src/sql_value.rs:681:21
|
681 | val.as_ptr() as *const i8,
| ^^^^^^^^^^^^^^^^^^^^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*const u8`
found raw pointer `*const i8`
error[E0308]: mismatched types
--> src/sql_value.rs:750:56
|
750 | chkerr!(self.ctxt(), dpiLob_writeBytes(lob, 1, ptr, len));
| ^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*const u8`
found raw pointer `*const i8`
error[E0308]: mismatched types
--> src/sql_value.rs:762:56
|
762 | chkerr!(self.ctxt(), dpiLob_writeBytes(lob, 1, ptr, len));
| ^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*const u8`
found raw pointer `*const i8`
error[E0308]: mismatched types
--> src/statement.rs:477:56
|
477 | dpiStmt_getBindNames(handle, &mut num, names.as_mut_ptr(), lengths.as_mut_ptr()),
| ^^^^^^^^^^^^^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*mut *const u8`
found raw pointer `*mut *const i8`
error[E0308]: mismatched types
--> src/statement.rs:484:45
|
484 | bind_names.push(to_rust_str(names[i], lengths[i]));
| ^^^^^^^^ expected `u8`, found `i8`
|
= note: expected raw pointer `*const u8`
found raw pointer `*const i8`
For more information about this error, try `rustc --explain E0308`.
Getting this while doing select, I beleive it is :
ORA-01406: fetched column value was truncated
Cause: In a host language program, a FETCH operation was forced to truncate a character string. The program buffer area for this column was not large enough to contain the entire string. The cursor return code from the fetch was +3.
Action: Increase the column buffer area to hold the largest column value or perform other appropriate processing.
Any ideas how to fix?
Dear Kubo-san,
First of all, thanks for the great wrapper library!
However, there is a small issue with Object::get_by_attr method: when using FLOAT data type, a call to dpiObject_getAttributeValue results in a segfault, because internally ODPI-C library treats FLOAT as a subtype of NUMBER and expects a buffer in asBytes.ptr, which is currently set only when the attr.oratype is OracleType::Number (.. which corresponds to NUMBER, but not to FLOAT).
I am unsure whether it is a bug in oracle's documentation of dpiObject_getAttributeValue which only mentions DPI_ORACLE_TYPE_NUMBER or not, but nevertheless is seems like we need to utilize dpiData_setBytes also for attr.oratype being OracleType::Number.
We made a local circumvention by migrating from FLOAT
to NUMBER
in our database, so unfortunately we can't test whether adding a simple if
condition for OracleType::Float
would work, hence we are not providing a PR, sorry :(
it seems that currently optional features (chrono
& aq_unstable
at this time) are not being tested by the CI workflow. i'd suggest adding a build matrix which covers also these features.
see e.g. what i did in rursprung/r2d2-oracle#15 for using a build matrix with features. with #58 i'm adding a build matrix to this repo here anyway, so maybe you could then directly extend that and just add the features to the matrix as well (so that they're also tested with both versions of rust and on all OS). you might want to test all four combinations (no features, all features and each feature individually) or just two of them: all features and no features.
Hi,
It we would be great to be able to check if the context initialized successfully (I had issues with LD_LIBRARY_PATH). The problem arise at Connector::connect().unwrap()
Is it possible to check if the context is properly created before attempting a connection?
It seems like the Context
struct is private, but I may have missed something.
What do you think about it?
My Cargo.toml:
[dependencies]
oracle = "*"
My code:
extern crate oracle;
use oracle::{Connection, Error, Result};
fn sql()-> Result<()> {
// Connect to a database.
let conn = Connection::connect("scott", "tiger", "//base/base", &[])?;
let sql = "select ename, sal, comm from emp where deptno = :1";
// Select a table with a bind variable.
println!("---------------|---------------|---------------|");
let rows = conn.query(sql, &[&30])?;
for row_result in &rows {
let row = row_result?;
// get a column value by position (0-based)
let ename: String = row.get(0)?;
// get a column by name (case-insensitive)
let sal: i32 = row.get("sal")?;
// Use `Option<...>` to get a nullable column.
// Otherwise, `Err(Error::NullValue)` is returned
// for null values.
let comm: Option<i32> = row.get(2)?;
println!(" {:14}| {:>10} | {:>10} |",
ename,
sal,
comm.map_or("".to_string(), |v| v.to_string()));
}
// Another way to fetch rows.
// The rows iterator returns Result<(String, i32, Option<i32>)>.
println!("---------------|---------------|---------------|");
let rows = conn.query_as::<(String, i32, Option<i32>)>(sql, &[&10])?;
for row_result in &rows {
let (ename, sal, comm) = row_result?;
println!(" {:14}| {:>10} | {:>10} |",
ename,
sal,
comm.map_or("".to_string(), |v| v.to_string()));
}
Ok(())
}
fn main() {
match sql() {
Ok(_) => {},
Err(error) => { println!("{:?}", error); }
}
}
The code is compiled ok. I am compiling to the windows gnu target:
cargo build --release --target i686-pc-windows-gnu
But the execution result is:
DpiError(DbError { code: 0, offset: 0, message: "DPI-1020: version 3.0 is not supported by ODPI-C library version 2.4", fn_name: "dpiContext_create", action: "check version" })
i'd strongly suggest to move to using Semantic Versioning for this repository to make it easier for consumers to use it.
e.g. somewhere between 0.5.1 and 0.5.5 (presumably with 0.5.5 itself) support for rustc 1.42.0 has been removed (the minimum version required now is 1.54.0), so this should've been a 0.6.0 release instead.
using semantic versioning will clearly tell consumers what to expect when they update to a new version. also, the whole dependency version management system of cargo is built around semantic versioning (see the docs).
on a related note i'd suggest to move your changelog to follow Keep a Changelog to keep things more standardised (your changelog is already pretty closely aligned with this and will only need some minor enhancements for it to follow the standard).
I would like to be able to set the properties DEFAULT_LOBPREFETCH_SIZE and DEFAULT_ROW_PREFETCH when connections are created (incl. pooled connections from r2d2_oracle): https://docs.oracle.com/cd/E18283_01/appdev.112/e10646/ociaahan.htm#sthref5292
Hi!
I want to share what I found out with today's debugging.
I am preparing a simple rust program with http server.
In the beginning I am creating an "global" oracle connection and wrap it into:
Arc<Mutex<oracle::Connection>>
then on every async http request I am doing this sample code:
[...]
let conn = odata.lock()?
let sql = "begin erp.my_pkg.convert(:1, :2, :3, to_date(:4, 'YYYY-MM-DD HH24:MI:SS'), :5, NULL, :6, :7, :8, :9); end;";
let mut stmt = conn.prepare(sql, &[])?;
stmt.execute(&[&id, &arg2, &id_rec, &datetime, &reader_no, &id_loc, &mode, &search, &OracleType::Varchar2(32767)])?;
let out_id: i32 = stmt.bind_value(1).unwrap_or(-1);
let outval: String = stmt.bind_value(9).unwrap_or_default();
conn.commit()?;
This is working great, but when our dba intentionally close the session on the server side to test my program then my program suddenly stops with:
tokio-runtime-w[2998]: segfault at 10 ip 000055c31cfc9ade sp 00007fdcac660d00 error 4 in injector[55c31ce47000+4c7000]
so it is segfaulting without any panic()
Then I did some research and the offset in which it segfaults is:
So it is inside dpiGen__setRefCount()
, which we can see eg here:
https://github.com/oracle/odpi/blob/master/src/dpiGen.c#L265
Maybe you are able to reproduce it? It is a little hard for me to reproduce as I am not the owner of this oracle server...
Do you think it is a rust-oracle bug? Can you report it upstream in other case?
I think something is wrong because instead of rust panic it is just segfaulting...
Hi,
Environment:
When setting privileges, CTRL-C doesn't work anymore.
Reproducible example:
use oracle::{Connector, Privilege};
use std::thread;
use std::time::Duration;
fn main() {
let connector = Connector::new("sys", "sys", "localhost:1521/orcl.localdomain")
//.privilege(Privilege::Sysdba) <-- uncomment to trigger the issue
.to_owned();
let conn = connector.connect();
loop {
println!("try to CTRL-C! while I sleep...");
thread::sleep(Duration::from_millis(5_000));
}
}
I'm not exactly sure why, but the signals handlers are setup differently.
Below is the output of strace -e rt_sigaction,rt_sigprocmask ./target/release/main 2>&1 | grep INT
no privileges - does work
rt_sigprocmask(SIG_BLOCK, [INT], NULL, 8) = 0
rt_sigaction(SIGINT, {sa_handler=0x7f9ed5c54840, sa_mask=~[ILL TRAP ABRT BUS FPE SEGV USR2 TERM XCPU XFSZ SYS RTMIN RT_1], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7f9edc35bb30}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [INT], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT], NULL, 8) = 0
rt_sigaction(SIGINT, {sa_handler=SIG_DFL, sa_mask=~[ILL TRAP ABRT BUS FPE SEGV USR2 TERM XCPU XFSZ SYS RTMIN RT_1], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7f9edc35bb30}, {sa_handler=0x7f9ed5c54840, sa_mask=~[ILL TRAP ABRT BUS FPE KILL SEGV USR2 TERM STOP XCPU XFSZ SYS RTMIN RT_1], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7f9edc35bb30}, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [INT], NULL, 8) = 0
privileges - does not work
rt_sigprocmask(SIG_BLOCK, [INT], NULL, 8) = 0
rt_sigaction(SIGINT, {sa_handler=0x7f9ffb02d840, sa_mask=~[ILL TRAP ABRT BUS FPE SEGV USR2 TERM XCPU XFSZ SYS RTMIN RT_1], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7fa001734b30}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [INT], NULL, 8) = 0
Any idea?
The current design of oracle::Statement
makes it hard to impossible to implement a efficient prepared statement mechanism on top of the oracle
crate.
A common strategy to implement prepared statement caching is to bundle a connection and the associated statement cache into a common data structure. So any downstream crate that tries to provide a querybuilder + prepared statement caching will probably want to write code similar to this.
struct OracleConnection {
inner: oracle::Connection,
stmt_cache: StmtCache<oracle::Statement>,
}
Given that oracle::Statement<'a>
has a lifetime 'a
that is coupled to the connection type itself such a pattern is hardly possible, as rust does support self referential types poorly.
(Hit this while trying to provide a diesel connection implementation on top of this crate)
My environment is windows10.
rust's version is 1.35.0(stable-msvc).
crate oracle 0.2.1
My application hangs up due to out of memory.
I watched details of memory usage when my application hangs up.
It seems memory and handles are not release.
The following is a simplification of my code.
To create threads, then individual thread connects oracle and executes query.
After long time running, out of memory is going to happen.
Allocated memory never release and reuse.
Is my code somthing wrong?
(I modified previous post.)
extern crate oracle;
use oracle::Connection;
use std::thread;
fn main() {
let limit = 1000;
thread::spawn(move || {
for _ in 0..limit {
let h = thread::spawn(move || {
let conn = Connection::connect(
"user",
"pass",
"192.168.1.1:1521/orcl",
&[]).unwrap();
conn.close().unwrap();
});
h.join().unwrap();
}
}).join().unwrap();
println!("finished");
let mut word = String::new();
std::io::stdin().read_line(&mut word).ok();
}
I have multiple f32 stored together as blob. it there a way to read the raw blob. or even better to read the f32 to a vec.
I figure out sqlValue.get<f32>() should get a single one, but I have multiple of them. Thanks.
Run the demo(select.rs) , the error when build:
error[E0277]: `oracle::ResultSet<'_, oracle::Row>` is not an iterator
--> src/main.rs:13:23
|
13 | for row_result in rows {
| ^^^^ `oracle::ResultSet<'_, oracle::Row>` is not an iterator
|
= help: the trait `std::iter::Iterator` is not implemented for `oracle::ResultSet<'_, oracle::Row>`
= note: required by `std::iter::IntoIterator::into_iter`
error: aborting due to previous error
ENV:
rustc 1.36.0-nightly (938d4ffe1 2019-04-27)
I have an error on win10:
running: "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64\cl.exe" "/nologo" "/MD" "/Z7" "/I" "odpi/include" "/I
" "odpi/src" "/W4" "-Wno-unused-parameter" "/FoC:\Rust\rms\target\debug\build\oracle-a41627ac717eac75\out\odpi/src\dpiConn.o"
"/c" "odpi/src/dpiConn.c"
cargo:warning=cl : Command line error D8021 : invalid numeric argument '/Wno-unused-parameter'
exit code: 2
To fix this error I removed line .flag("-Wno-unused-parameter") from build.rs. And now it works for me.
Object type cache in connection prevents connection drop, since ObjectTypeInternal
inside InnerConn
have an Conn (Arc<InnerConn>)
to this connection, creating cycle.
Example below will always create new connections without dropping previous, leading to memory and socket leaks. This can be checked with $ sudo netstat -tnp | grep <app_name> | grep <port> | grep ESTAB | wc -l
. Uncommenting //conn.close().unwrap();
won't help - socket will be closed, but memory will still leak. Also this can be checked with $ DPI_DEBUG_LEVEL=95 cargo run
- ODPI logs will show that old connection is not released.
fn main() {
let connector = oracle::Connector::new("user", "password", "conn_str");
loop {
let conn = connector.connect().unwrap();
drop(conn.object_type("some_type").unwrap());
//conn.close().unwrap();
std::thread::sleep(std::time::Duration::from_secs(3));
}
}
I have not found one yet, since there is many Arc's in library, which references same connection from different places, so I doubt this can be easily fixed without many changes. At least it is good idea to document this behavior.
Hey, sometimes requests to my db take quite a long time, I'd like to be able to drop the connection and cancel all the requests that are going on somehow - could I do this? great job on this crate btw, it works like a charm, cheers
When I try to run select round(v, 2) from test_round;
, I want to get Float number result, but the return type is Number(0,0).
Number(0,0) will be parsed as Integer because the scale is 0.
test_round
v
1.111
2.222
3.333
Would be nice to have support for at least one of these: LONG, LONG RAW and BLOB data type, nowadays is very common to store binary data in the DB (Megabytes or Gigabytes).
Any chance this can be done in the short term?
Regards,
R
Hello,
I've tried to use this library and it works in debug builds. But when I compile them with cargo build --release
, I'll get following panic.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: OCI Error: (code: 0, offset: 0, message:, fn_name: dpiConn_create, action: start)', /checkout/src/libcore/result.rs:906:4
It happens when calling oracle::Connection::new
- I've reduced my code to basically creating connection in main
.
rustc 1.23.0-beta.2 (c9107ee93 2017-12-08)
on ubuntu
I'm just learning rust, so I'm not sure if I can be more helpful with debugging, but I'll try if needed.
The connection ping fn seems to never return when no connection to db is present. The sample code below hangs during the match for conn.ping() and never returns Ok nor Err
I was attempting to use the ping to check if the database is still available because we are having some network issues. Am I missing something here?
Sample code:
let conn = match Connection::connect(username, password, "localhost:1521/DB1") {
Ok(c) => {c}
Err(err) => {panic!("{:?}", err)}
};
let delay = std::time::Duration::from_secs(3);
loop {
match conn.ping() {
Ok(_) => { log::info!("Ping OK!"); }
Err(err) => { log::info!("Ping Err: {:?}", err); }
};
std::thread::sleep(delay);
}
First off, I don't think this is the fault of rust-oracle. It seems to be either OCI or the way ODPI-C uses OCI but I'd appreciate your opinion.
I have an application which, using many CPU threads, reads all data from all tables in a database schema to generate a SHA256 checksum of each table. I would expect the CPU to be maxing out with the SHA256 functions but when profiling the application most of the time seems to be taken up with allocating memory for table data and freeing it after use.
I've made a small reproduction program at https://github.com/daviessm/rust_oracle_performance which demonstrates the issue. Although it discards all the data rather than hashing it, I would hope for much better performance. The flamegraph at https://raw.githubusercontent.com/daviessm/rust_oracle_performance/master/flamegraph.svg shows 28% of the runtime is in dpiGen__release
and 31% is in dpiConn_newVar
.
To run the program, use the parameters -d
for database connection string (e.g. 127.0.0.1/orcl
), -u
for username and -p
for password. On the first run it'll create a table called test1
which contains 50 columns and 2,000,000 rows of VARCHAR2 data, then reads from it using one thread for each CPU on the client. Subsequent runs will use the existing table.
The following code gives the error[E0106]: missing lifetime specifier at <ResultSet>.
'''use oracle::{Connection, ResultSet, Row};
fn oracle_connect(uid: &str, pass: &str, server: &str, sql: &str) -> oracle::Result<ResultSet> {
let conn = Connection::connect(uid, pass, server)?;
let mut stmt = conn.prepare(sql, &[])?;
stmt.query(&[])
}
'''
According to the manual this is how the return type is written but it doesn't work.
I am looking to hook up the my rust app to oracle database and we use AQ for communication. I can see in the bindings that there are functions for enq/deq and also for subscriptions, but can not find any examples on how to use them or any rust struct in form of a Queue.
Is this not (yet) supported or am I missing something?
in the TODO list of your README you mention connection pooling. what are your plans there? i provide r2d2-oracle
which wires up your oracle
with the r2d2
connection manager.
is this enough? maybe it'd make sense to strike this list from your TODO and instead point to r2d2-oracle
in your README?
@kubo apologies if you already have this queued up from the email I sent about it.
I think the comment in connection.js about prefetch size could should show a value one greater than the number of rows. I.e this:
/// let mut stmt = conn
/// .statement("select empno, ename from emp order by empno fetch first 10 rows only")
/// .prefetch_rows(10)
/// .build()?;
could be:
/// let mut stmt = conn
/// .statement("select empno, ename from emp order by empno fetch first 10 rows only")
/// .prefetch_rows(11) /// add one to avoid a round-trip to check for end-of-fetch
/// .build()?;
It depends what cases you want to show.
Dear Sir,
I want to get EMP info when ename is either scott or june or kate etc. , how to bind it ?
eg:
let sql = "select * from emp from ename in (:1)";
let ename = ...; // how to set it ?
let rows = conn.query(sql, &[&ename])?;
I'm getting a compilation failure when pulling in oracle. I've tried using the master branch:
oracle = { git = "https://github.com/kubo/rust-oracle.git", features = ["chrono"] }
And version 0.3.1:
oracle = { version = "0.3.1", features = ["chrono"] }
rustc
version:
stable-x86_64-apple-darwin unchanged - rustc 1.39.0 (4560ea788 2019-11-04)
error[E0277]: `*const i8` cannot be sent between threads safely
--> /Users/nathansculli/.cargo/git/checkouts/rust-oracle-858dd8ce065f6a17/2e61902/src/lib.rs:411:1
|
411 | / lazy_static! {
412 | | static ref DPI_CONTEXT: ContextResult = {
413 | | let mut ctxt = Context {
414 | | context: ptr::null_mut(),
... |
449 | | };
450 | | }
| |_^ `*const i8` cannot be sent between threads safely
|
= help: within `ContextResult`, the trait `std::marker::Send` is not implemented for `*const i8`
= note: required because it appears within the type `binding::dpiErrorInfo`
= note: required because it appears within the type `ContextResult`
= note: required because of the requirements on the impl of `std::marker::Sync` for `spin::once::Once<ContextResult>`
= note: required because it appears within the type `lazy_static::lazy::Lazy<ContextResult>`
= note: shared static variables must have a type that implements `Sync`
= note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
error: aborting due to previous error
For more information about this error, try `rustc --explain E0277`.
error: could not compile `oracle`.
Hi, after spending a few hours and being unable to configure the PATH or environment variable for Oracle Instant Client, I decided to ask for your help here as the last resort. Has anyone configured whatever is required for Oracle Instant Client on MacOs Big Sur?
This is the error I'm getting:
DPI Error: DPI-1047: Cannot locate a 64-bit Oracle Client library: "dlopen(libclntsh.dylib, 1): image not found". See https://oracle.github.io/odpi/doc/installation.html#macos for help
I've been trying to set the following, but whatever I tried has not worked:
Alternatively, set DYLD_LIBRARY_PATH to the Instant Client directory. Note this variable does not propagate to sub-shells.
Hi! Thanks for the library. Is it possible to statically link the oracle instant client?
According to this stackoverflow post about the same in go it seems possible, but I'm new to linking binaries.
What I have tried so far?
So we can access the rows of the result set of call like conn.query_named in an async function.
Then call this async function with tokio spawn.
Thanks.
Hi, How can I query varray type data to a struct and then convert it to vector of string?
Thanks!
Is it possible to support failover - with multiple server endpoints in a connection string?
Hi,
We try to use rust-oracle for call function that returns an object and get Oracle Error.
This is the example of errors encountered in Rust Oracle Crate
ORA-06502: PL/SQL: numeric or value error
While trying to execute a query INTO object.
If return data contains extra long fields (with unspecified length boundary of 200-300-500 UTF chars) error is produced by connection driver.
In case the same query is executed via other DB client(like DBeaver) result is displayed in full with no errors raised.
Notes:
The response overall length somehow affects this behaviour.
If Only one field (Attribute A) has extra long data, but still matches the unspecified maximum length boundary, it appears to return correctly.
If there are other fields in response (Attributes B C D), summarised length of all fields is compared against unspecified char length limit.
Steps to reproduce
1.Create database objects
CREATE OR REPLACE TYPE T_OBJ_LONG AS OBJECT (
data1 VARCHAR2(500 CHAR),
data2 VARCHAR2(500 CHAR),
data3 VARCHAR2(500 CHAR),
data4 VARCHAR2(500 CHAR)
) ;
CREATE OR REPLACE FUNCTION t_test_long(v_count IN NUMBER) RETURN T_OBJ_LONG
IS
v_res T_OBJ_LONG;
BEGIN
v_res := T_OBJ_LONG(
dbms_random.string('U',v_count),
dbms_random.string('U',v_count),
dbms_random.string('U',v_count),
dbms_random.string('U',v_count)
);
RETURN v_res;
END;
use oracle::{ Connection, sql_type, Error };
use oracle::sql_type::{ObjectType, Object};
fn main() {
static SQL_TEST:&str = "CALL t_test_long(:v_count) INTO :v_result ";
let conn: Connection = Connection::connect("[USER]", "[PASS]", "//[SERVER]:[PORT]/[SERVICE NAME]").unwrap();
let mut result_type = conn.object_type("T_OBJ_LONG").unwrap();
let mut stmt = conn.prepare(SQL_TEST, &[]).unwrap();
//Everythig works fine when we return 4 string with 10 chars each
stmt.execute(&[
&10,
&sql_type::OracleType::Object(result_type)]
).unwrap();
let v_result: sql_type::Object = stmt.bind_value("v_result").unwrap();
println!("{:?} + {:?} + {:?} + {:?}",
v_result.get::<String>("DATA1").unwrap().len(),
v_result.get::<String>("DATA2").unwrap().len(),
v_result.get::<String>("DATA3").unwrap().len(),
v_result.get::<String>("DATA4").unwrap().len(),
);
// Reproduce ORA-06502: PL/SQL: numeric or value error
// Executing a query with extra long data return (500 chars)
result_type = conn.object_type("T_OBJ_LONG").unwrap();
stmt.execute(&[
&500,
&sql_type::OracleType::Object(result_type)]
).unwrap();
let v_result: sql_type::Object = stmt.bind_value("v_result").unwrap();
println!("{:?} + {:?} + {:?} + {:?}",
v_result.get::<String>("DATA1").unwrap().len(),
v_result.get::<String>("DATA2").unwrap().len(),
v_result.get::<String>("DATA3").unwrap().len(),
v_result.get::<String>("DATA4").unwrap().len(),
);
}
Running `target/debug/rust_oracle_issue`
10 + 10 + 10 + 10
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: OciError(DbError { code: 6502, offset: 0, message: "ORA-06502: PL/SQL: numeric or value error", fn_name: "dpiStmt_execute", action: "execute" })', src/libcore/result.rs:1084:5
stack backtrace:
0: backtrace::backtrace::libunwind::trace
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.34/src/backtrace/libunwind.rs:88
1: backtrace::backtrace::trace_unsynchronized
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.34/src/backtrace/mod.rs:66
2: std::sys_common::backtrace::_print
at src/libstd/sys_common/backtrace.rs:47
3: std::sys_common::backtrace::print
at src/libstd/sys_common/backtrace.rs:36
4: std::panicking::default_hook::{{closure}}
at src/libstd/panicking.rs:200
5: std::panicking::default_hook
at src/libstd/panicking.rs:214
6: std::panicking::rust_panic_with_hook
at src/libstd/panicking.rs:477
7: std::panicking::continue_panic_fmt
at src/libstd/panicking.rs:384
8: rust_begin_unwind
at src/libstd/panicking.rs:311
9: core::panicking::panic_fmt
at src/libcore/panicking.rs:85
10: core::result::unwrap_failed
at src/libcore/result.rs:1084
11: core::result::Result<T,E>::unwrap
at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/result.rs:852
12: rust_oracle_issue::main
at src/main.rs:34
13: std::rt::lang_start::{{closure}}
at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/rt.rs:64
14: std::rt::lang_start_internal::{{closure}}
at src/libstd/rt.rs:49
15: std::panicking::try::do_call
at src/libstd/panicking.rs:296
16: __rust_maybe_catch_panic
at src/libpanic_unwind/lib.rs:80
17: std::panicking::try
at src/libstd/panicking.rs:275
18: std::panic::catch_unwind
at src/libstd/panic.rs:394
19: std::rt::lang_start_internal
at src/libstd/rt.rs:48
20: std::rt::lang_start
at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/rt.rs:64
21: main
22: __libc_start_main
23: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
OS : Linux
Oracle : Oracle Database 12c Standard Edition Release 12.2.0.1.0 - 64bit Production
Client : oracle-instantclient19.3-basic-19.3.0.0.0-1.x86_64
Implementations of RowValue
for structs is usually simple and all similar, so it make sense to make #[derive(RowValue)]
macro (in our case we had struct with over 150 fields). We currently have our own implementation of this procedural macro and we want to have it in rust-oracle
. Do you interested in PR in this repo (with proc-macro crate, enabled by feature) or should we publish it separately?
#[derive(RowValue)]
struct Row {
id: i64,
#[row_value(rename = "timestamp")
ts: NaiveDateTime,
#[row_value(with = "custom::parse::fn")]
name: Option<String>,
}
Hi
i have just published v0.1.0 of the r2d2-oracle library: https://github.com/rursprung/r2d2-oracle
could you please have a look at it and check if it looks good to you?
Thanks!
My problem now is that some of the connections just hang indefinitely.
my code:
#[instrument(skip_all)]
async fn acquire_connection(
OracleConnectionCredentials {
username,
password,
connect_string,
}: OracleConnectionCredentials,
) -> Result<Arc<Connection>> {
tokio::task::spawn_blocking(move || {
debug!("getting connection out of a pool");
let conn = oracle::Connector::new(username, password, connect_string)
.purity(oracle::conn::Purity::New)
.connect()
.map(Arc::new);
conn
})
.await
.wrap_err("thread crashed")
.and_then(|v| v.wrap_err("acquiring"))
}
can I either give it some timeout or break this process externally?
I can't compile your example with these errors:
error[E0277]: the ?
operator can only be used in a function that returns Result
or Option
(or another type that implements Try
)
--> src\main.rs:7:16
|
4 | / fn main() {
5 | |
6 | | // Connect to a database.
7 | | let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;
| | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot use the ?
operator in a function that returns ()
... |
29 | | }
30 | | }
| |_- this function should return Result
or Option
to accept ?
|
= help: the trait Try
is not implemented for ()
= note: required by from_error
error[E0277]: the ?
operator can only be used in a function that returns Result
or Option
(or another type that implements Try
)
--> src\main.rs:13:16
|
4 | / fn main() {
5 | |
6 | | // Connect to a database.
7 | | let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;
... |
13 | | let rows = conn.query(sql, &[&30])?;
| | ^^^^^^^^^^^^^^^^^^^^^^^^ cannot use the ?
operator in a function that returns ()
... |
29 | | }
30 | | }
| |_- this function should return Result
or Option
to accept ?
|
= help: the trait Try
is not implemented for ()
= note: required by from_error
error[E0277]: the ?
operator can only be used in a function that returns Result
or Option
(or another type that implements Try
)
--> src\main.rs:15:19
|
4 | / fn main() {
5 | |
6 | | // Connect to a database.
7 | | let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;
... |
15 | | let row = row_result?;
| | ^^^^^^^^^^^ cannot use the ?
operator in a function that returns ()
... |
29 | | }
30 | | }
| |_- this function should return Result
or Option
to accept ?
|
= help: the trait Try
is not implemented for ()
= note: required by from_error
error[E0277]: the ?
operator can only be used in a function that returns Result
or Option
(or another type that implements Try
)
--> src\main.rs:17:29
|
4 | / fn main() {
5 | |
6 | | // Connect to a database.
7 | | let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;
... |
17 | | let ename: String = row.get(0)?;
| | ^^^^^^^^^^^ cannot use the ?
operator in a function that returns ()
... |
29 | | }
30 | | }
| |_- this function should return Result
or Option
to accept ?
|
= help: the trait Try
is not implemented for ()
= note: required by from_error
error[E0277]: the ?
operator can only be used in a function that returns Result
or Option
(or another type that implements Try
)
--> src\main.rs:19:24
|
4 | / fn main() {
5 | |
6 | | // Connect to a database.
7 | | let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;
... |
19 | | let sal: i32 = row.get("sal")?;
| | ^^^^^^^^^^^^^^^ cannot use the ?
operator in a function that returns ()
... |
29 | | }
30 | | }
| |_- this function should return Result
or Option
to accept ?
|
= help: the trait Try
is not implemented for ()
= note: required by from_error
error[E0277]: the ?
operator can only be used in a function that returns Result
or Option
(or another type that implements Try
)
--> src\main.rs:23:33
|
4 | / fn main() {
5 | |
6 | | // Connect to a database.
7 | | let conn = Connection::connect("scott", "tiger", "//localhost/XE")?;
... |
23 | | let comm: Option = row.get(2)?;
| | ^^^^^^^^^^^ cannot use the ?
operator in a function that returns ()
... |
29 | | }
30 | | }
| |_- this function should return Result
or Option
to accept ?
|
= help: the trait Try
is not implemented for ()
= note: required by from_error
error: aborting due to 6 previous errors; 1 warning emitted
I am using Rust 1.52.1
This happens randomly, I'm not getting it when executing the same queries using say DataGrip
From oracle/odpi#163 - this would allow CLOBs to be accessed as strings, rather than LOBs through LOB locators.
I need to get the value of ROWIDs selected from a table in the format select rowid from table1
but the Rowid
type doesn't seem to be castable to any type in Rust - please could you document in the FromSql
trait how I can reference it as a string (as would be returned by ROWIDTOCHAR
)?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.